Second screen subtitles function

ABSTRACT

Systems for, and methods of, displaying video information comprising: a second screen device obtaining current play position data of a video being played on a primary screen device (e.g., obtaining from the primary screen device an identification of a current play position of the video, or obtaining information to generate an acoustic fingerprint of the video); determining a current play position of the video playing on the primary screen device based upon the current play position data (e.g., identification of the current play position or the acoustic fingerprint); downloading information (e.g., video map, subtitles, moral principles, objectionable content, memorable content, performers, geographical maps, shopping, plot point, item, ratings, and trivia information) over a computer communications network into the memory of the second screen device; and displaying information on the second screen device synchronized with the contemporaneously played video on the primary screen device.

RELATED U.S. APPLICATION DATA

This application is a continuation-in-part of patent application Ser.No. 13/964,176, filed Aug. 12, 2013, which is a continuation-in-part ofpatent application Ser. No. 13/506,093, filed Mar. 26, 2012.

BACKGROUND OF THE INVENTION

1. Field of the Invention

Systems for, and methods of, displaying, on a second screen, during aplaying of a video on a primary screen, information relating to acurrent position in the video. The synchronization of the processing ofthe information to the playing of the movie on the primary screencomprises, for example, a timecode retrieval from the video or anacoustic fingerprint matching. The processing and/or retrieval ofinformation may be at the second screen, local server, remote server,and or services provider.

2. Description of the Related Art

Systems for, and methods of, displaying on a second screen, during aplaying of a video on a primary screen, information relating to acurrent position in the video are limited in the functions supported andthe capabilities and information that are provided the user.

BRIEF SUMMARY OF THE INVENTIONS

The present inventions relate generally to systems and methods forproviding supplementary information on a second screen during a playingof a video on a primary screen.

Accordingly, it is an object of the present inventions to, for example,provide user capabilities on a second screen comprising routines for:establishing content preferences without interrupting the viewing byother users of a playing of a video on a primary screen; contentpreviewing that, responsive to a preference with respect to a level ofexplicitness in one of a plurality of content categories, enablespreviewing yet to be played segments within a video; enabling the userto search video segments depicting a performer/character having certainphysical attributes and performing specified acts; displaying multiplegeographical map views responsive to a depicted locale, and forproviding linkages to secondary information; displaying plot informationand for enabling the user to establish preferences as to a notificationwith respect to categories of plot information (e.g., disablingnotification of clues; identifying moral principles of a moral dilemmadepicted in the video, and suggesting moral dilemma questions;displaying multiple website pages of a purchase item depicted in thevideo; notifying the proximity to a brick and mortar retailer of aproduct depicted in a video for which the user indicated an interest;the controlled synchronized delay display of subtitles; theidentification of the best lines, memorable moments, best performances,and romantic moments in a video and enabling a user to individually ratethem and share the ratings with other users; limiting a search of thecontent of a video that precedes the current play position, displayingan identification of items and trivia questions contemporaneously with aplaying of the video on a primary screen; and displaying, responsive toa user's content classification preferences, a notification of suitablepoints for a snack break.

Briefly, these and other objects are enabled by systems for, and methodsof, displaying video information comprising: a second screen deviceobtaining current play position data from a video being played on aprimary screen device (e.g., obtaining from the primary screen device anidentification of a current play position of the video, or obtainingaudio information to generate an acoustic fingerprint of the video);determining a current play position of the video playing on the primaryscreen device based upon the current play position data (e.g.,identification of the current play position or the acousticfingerprint); downloading information (e.g., video map, subtitles, moralprinciples, objectionable content, memorable content, performers,geographical maps, shopping, plot point, item, ratings, and triviainformation) over a computer communications network into the memory ofthe second screen device; and displaying information on the secondscreen device synchronized with the contemporaneously played video onthe primary screen device. These and other embodiments, features,advantages, and objects, are apparent in the context of the detaileddescription, accompanying drawings, and appended claims that follow.

BRIEF DESCRIPTION OF THE DRAWINGS

The following drawing figures, that form a part of this application, areillustrative of embodiments of the present invention and are not meantto limit the scope of the invention in any manner.

FIG. 1 is a schematic diagram of a video provider and end usercommunications infrastructure;

FIG. 2 is an illustration of a smartphone remote control interface;

FIG. 3A is an illustration of a second screen display of features of aWhat, ID, and Credits functions;

FIG. 3B is an illustration of offset synchronization methodologies;

FIG. 3C is an illustration of a second screen display of features of aWho function;

FIG. 4A is an illustration of a second screen display of features of aLocations function;

FIG. 4B is an illustration of a second screen display of a set ofgeographical maps;

FIG. 5 is an illustration of a second screen display of features of aPlot Info function;

FIG. 6 is an illustration of a second screen display of features of aDilemma function;

FIG. 7 is an illustration of a second screen display of features of aTrivia and notification functions;

FIG. 8A is an illustration of a second screen display of features of aShopping function;

FIG. 8B is an illustration of a second screen display of features of theShopping function simultaneous display of multi-sourced externalinformation;

FIG. 9A is an illustration of a second screen interface for establishingcontent preferences;

FIG. 9B is an illustration of a second screen interface for establishingcontent preferences and previewing content;

FIG. 9C is an illustration of a second screen display of features of aRatings function;

FIG. 10A is an illustration of a second screen display of features of aSearch function;

FIG. 10B is an illustration of a second screen display of features of acontent-categorized Search function;

FIG. 11A is an illustration of a second screen display of features of afunction in a tiles mode;

FIG. 11B is an illustration of a second screen display of features of afunction in a list mode;

FIG. 11C is an illustration of a use of a touch screen to definespecific casting/mirroring of second screen content;

FIG. 12 is a flow chart of a method for displaying second screeninformation synchronized with video and notification display on aprimary screen; and

FIG. 13 is a flow chart of a method for displaying second screeninformation synchronized with video display on a primary screen.

DETAILED DESCRIPTION OF THE INVENTIONS

For purposes of the present disclosure, various terms used in the artare defined as follows:

The term “herein” shall mean in the entirety of this specificationincluding drawings, abstract, and claims. The term herein is not limitedto the paragraph or section in which it may appear.

The terms “include”, “comprise”, and “contains” do not limit theelements to those listed. By contrast, only the term “consist” limitsthe elements to those listed.

No conceptual distinction should be drawn from the use of the terms on,at, or in (e.g., no distinction should be drawn from the use of thephrase receiving on, receiving at, or receiving in a second screendevice).

The term “responsive” does not limit the elements, conditions,preferences, and/or requirements that may be taken into consideration.For example, an event that is responsive to a specified requirement isnot limited to being responsive to only that specified requirement. Anevent may be responsive to a specified requirement and a secondnon-specified requirement, specially, when the second requirement, whiledescribed as an alternative requirement, may be also deemedcomplementary.

The terms “application software”, “software application”, “application”,“app”, “routine”, and “computer software” shall mean all the executable,libraries, scripts, instructions, and/or processor executable steps inany format that causes, or is required by, a device to perform a task,function, or process. Application software or routines comprises acomputer program designed to assist a user to perform task, function,process, or activity. In some instances application software andoperating system software may be a synergistically integrated andindivisible.

The term “associate” shall mean assign, give, allocate, associate,designate, ascribe, attribute, link, and/or relate.

The term “clip” shall mean a segment that is smaller than a chapter andusually smaller than a scene. A clip includes one or more contiguousshots, and usually depicts the same primary characters within alocation. A clip's definition is responsive to a material change in theparticipation of the principal characters, a material change inlocation, and/or a distinct change in thematic content or topic ofconversation.

The term “descriptor” shall mean a keyword, word, term, code, phrase,designations, write-ups, and linkages. The term descriptor shall alsomean any data, information, image, and/or video frame that identifies,describes, links, and/or categorizes content of a video, portions of avideo, or a video frame. A linkage is any information, data, and/ormethod that enables retrieving and/or downloading data from alocal/internal and/or a remote/external source.

The term “dialog” shall mean a dialog, conversation, monologue, lyric,utterance, and/or communication. Dialog also comprises information thatmay be included in the subtitles or the closed captioning.

The term “geographic map” shall mean any map, including satellite,topographical, street, and such maps as Google Maps, Google Earth,Google Earth View, Google Street View, OpenStreetMap, and whether 2D or3D, static or dynamic and interactive, single or multi-featured, andrepresentative or photorealistic. The term “geographic map” shall alsomean any depiction (e.g., map) that provides context for a locale.

The term “item” shall mean: (i) an object, article, artifact,instrument, device, and product; (ii) a specific act or action within anactivity, process, event, and operation; (iii) an emotion, expression,gesture, and movement; (iv) an effect, consequence, and result; (v) asound, occasional foreign language, and melodic content; (vi) a portionof a dialog, line, and linguistic expression; (vii) cinematography,cinematographic technique, cinematographic effect, a special effect,technical transition, and production error; (viii) a cameo or specialappearance; and (ix) a locale.

The term “keywords” shall mean words, terms, phrases, designations,codes, descriptors, labels, data, metadata, and numbers.

The term “keywording” shall mean associating keywords.

The term “locale” shall mean a locale, location, site, setting, place,area, spot, landmark, location of interest, tourist attraction,building, building exterior, building interior, structure, and ageographically identifiable point. A locale is generally a location orplace outside of the movie studio that is used for filming a movie orportion of a movie. A locale may be depicted as the actual locale or maybe represented in the depictions as a locale other than the actuallocale. The term “locale” is differentiated from the term “location”when the term location refers to a point in the timeline of the video.

The term “navigator” shall mean application software and/or operatingsystem software that provide video playback capabilities, decoding,decrypting, and/or rendering, for playing a movie on a personalcomputer. A navigator comprises, for example, Microsoft's DVD Navigator,decoder filters, and renderer, to handle, for example, CSS and analogcopy protection.

The term “network” shall mean any private or public, wired or wirelesscommunication system.

The term “noteworthy” in connection with content shall mean contentthat: (i) may be of interest to a significant audience; (ii) isnoteworthy, remarkable, or compelling; and/or (iii) is uncommon,atypical, peculiar, unusual, unique, rare, or extraordinary.

The term “performer” shall mean an individual, participant, actor, oractress, appearing in a video and/or credited for the physical and/orthe verbal performance of a character. A performer refers to, forexample, an actor in a motion picture, an athlete in a televisedsporting event, a newscaster in a news program, and a chef in a cookingshow.

The terms “play” and “playing”, as in play or playing a segment of avideo, shall mean playing meaningfully, or substantially all or aportion of a segment. In other words, while a method or system disclosedherein may claim or intend to play the entirety of, or all, of asegment, a complete playing of a segment does not necessarily requirethe playing of every video frame, interlaced field, audio and subpicture portion, and/or bit of data of the segment.

The term “plot info” shall mean information, rationale, and/orexplanation relating to, or relevant to understanding or appreciating, aplot, sub-plot, theme, storyline, principle, idea, key point, clue,and/or item in the movie.

The term “plot point” shall mean a plot, sub-plot, theme, storyline,principle, idea, key point, clue, and item.

The term “preferences” shall mean “programming preference”, “versionpreference”, “presentation preference”, “content preferences”, “functionpreferences”, “technical preferences”, and “playback preferences”. Theterm “programming preference” shall mean a preference or preferences fora specific video (e.g. Spider-Man), genres of videos (e.g., Action),types of videos (e.g. interactive video detective games), series ofvideos (e.g., 007) broad subject matter of videos (e.g. mysteries),and/or time and date for playback of the video. The term “versionpreference” shall mean a preference or preferences for a version of avideo (e.g., motion picture), released by the copyright owner (e.g.,motion picture studio), that includes content not available in analternate version of the video. The version of a video refers to, forexample, the “Theatrical”, “Unrated”, and “Director's Cut” versionoptions in a DVD-Video. The version of a video does not refer to sequelsand/or remakes of a video such as Spider-Man (2002), Spider-Man 2 (2204)and The Amazing Spider-Man (2012). The term “presentation preference”shall mean a preference or preferences that cause the selectiveinclusion, in a presentation, of segments from within a video, from aversion of a video, or from within a plurality of videos. The term“presentation preference” shall also mean a preference or preferencesfor any one of the plurality of features provided by each of thefollowing: Presentations, Compilations, Subjects, Dilemmas, Best Of,Performers, Shopping, Music, Search, and Preview. The term “contentpreferences” shall mean preferences for the form of expression,explicitness, inclusion or exclusion of objectionable content, a levelof explicitness in each of a plurality of content categories of possiblyobjectionable content, length, level of detail, type of thematiccontent, and/or depictions of potentially objectionable items and/oracts. The Control feature of the CustomPlay application provides forcontent preferences. The term “function preference” shall mean apreference or preferences for any one of the plurality of elementsprovided by, or associated with, an in-video or playback function (e.g.,Who, What, Locations, Plot Info, Filmmaking, Trivia, and Info). The term“technical preference” shall mean a preference or preferences for thetechnical and/or artistic elements (e.g., dissolves, fades, and wipes)that may be implemented during the playing of non-sequential segments.The term “playback preference” shall mean a preference or preferencesfor visual and audio options (e.g., camera angles, picture with picture,subtitles, closed captioning, and commentaries) that may be availablefor a video.

The terms “seamless” and “seamlessly” shall mean without gapsperceptible to the human eye, achieved by maintaining a constant videotransmission rate. A seamless playing of non-sequential segments (i.e.,the skipping of a segment) while technically “seamless”, may not appearartistically seamless to a user because a change in the content that isplayed, rather than how it is played, suggested that a skip of contenttook place.

The term “search terms” shall mean terms, words, phrases, designations,codes, descriptors, labels, data, metadata, numbers, or otherinformation that identifies, describes, or specifies what is beingsearched.

The terms “second screen”, “secondary screen”, and “second screendevice” are interchangeable and shall mean any computing device capableof playing/displaying content, e.g., video, audio, images, subtitles,data, and/or information, and used in combination with a primary screendevice also capable of playing/displaying content. A primary screendevice is also referred to herein as a primary screen. A primary screenand a second screen includes, for example, televisions, personalcomputers, laptop and portable computers, tablets, smartphones, andmobile devices, remote control devices, and computing devices having adisplay screen. A primary screen device and a second screen device alsocomprises audio reproducing and outputting components (e.g., amplifiersand internal and/or external speakers).

The term “seek/step data” shall mean any index, data, and/or informationthat facilitates access to a video frame within a video and/orfacilitates the utilization of a video map with a video. Seek/Step dataneed not include step data (e.g., data informing a frame advance).Seek/step data may, without the step data, directly address every videoframe within a video. Further, for example, and not limitation,seek/step data need not be based on navigation points, synchronizinginformation (i.e., seek/step data) may be based on shot changes or scenechanges in the video.

The terms “segment” and “video segment” shall mean one or more videoframes. In the example of a feature presentation (e.g., Sixty minutes) asegment definition generally comprises an identification of a beginningand ending points (e.g., frames) within a video. However, in theexamples of second screen function implementations, a segment definitiongenerally identifies a single point (e.g., frame) within a video.

The term “subtitles” shall mean subtitles, and/or any textualinformation representative of a portion or portions of a video's audiodialogue. A display of subtitles does not require a display of all thesubtitles of a video. A display of subtitles may only display a subtitleportion, line, phrase, or unit. Herein, subtitles are materiallydistinct from closed captioning

The term “subtitle information” shall mean information (e.g., data,text, and/or images) that enables displaying subtitles on a screen.Embodiments that are detailed with respect to the display of subtitlesand/or use of subtitle information may be alternatively or complementarydisplay and/or use other supplementary information.

The term “supplementary information” shall mean any information, text,data, depiction, images, video, and/or content, that informs,entertains, elucidates, illuminates, illustrates, clarifies, and/orexplains.

The term “trailer” shall mean a trailer, preview, video clip, stillimage, and/or other content that precedes and/or is extraneous to themovie.

The term “user” is interchangeable with the terms “subscriber”,“viewer”, and “person”, and shall mean an end-user person actively usingvideo content, passively viewing a video, interactively playing a videogame, retrieving video from a video provider, and/or activelysubscribing to and using multimedia, internet, and/or communicationservices.

The term “variable content video” shall mean a video characterized by anonlinear architecture facilitating a variety of possible logicalsequences of segments. A variable content video comprises parallel,transitional, and/or overlapping segments to provide multiple versionsof a video. Responsive to the particular embodiment implemented, avariable content video may also include a user interface, applicationsoftware, software program routines, system control codes forcontrolling the playing of the video/audio, video map, bookmarkgenerating data, seek/step data, and/or map synchronization data. Avideo that does require parallel, transitional, and/or overlappingsegments to be variably played.

The terms “video”, and “video program” are interchangeable and shallmean any video image regardless of the source, motion, or technologyimplemented. A video may comprise images and audio found in full motionpicture programs, films, movies, interactive electronic games,multi-media content, television programs, commercials, advertisements,and/or entertaining, instructional, and/or educational programming.Responsive to the particular embodiment implemented, subtitles, subpicture information, user interface, application software, softwareprogram routines, system control codes for controlling the playing ofthe video/audio, video map, bookmark instructions, seek/step data,and/or map synchronization data may be provided with a video. The term“movie” shall mean a full length motion picture usually released intheaters and/or optical discs (e.g., a DVD-Video or Blu-ray Disc).

The terms “video map”, “map”, and “segment map”, shall mean anycombination, arrangement, table, database, listing, index, and/orinformation that: defines a beginning and ending of one or moresegments; identifies one or a plurality of individual video frames;and/or identifies one or a plurality of play positions in a video or theaudio of a video. A video map further comprises data associated with atleast: one segment, a sequence of segments, a video frame, and/or a playpositions in a video or the audio of a video. A video map's data maycomprise, for example: (i) a descriptor; (ii) an implicit or explicitediting and/or filtering action; (iii) a linkage among segments; (iv) alinkage to an internal/external source of information/content; (v) data,textual, image, audio, and/or video content; and/or (vi) suchinformation, data, linkages, and content that may required to enable orsupport the features and functions detailed herein. A video map mayfurther comprise bookmark generating data, seek/step data, and videosynchronizing information (e.g., an acoustic signature matchingdatabase).

The above defined terms and other terms explicitly defined herein are tobe understood as defined in this document, and not as they may bedefined by incorporation. Incorporation by reference shall not act tomodify, limit, or broaden the definitions hereinabove provided orformally defined in this document. A term that is not formally definedin this document is defined herein to have its ordinary and customarymeanings.

Networks and End-User Systems

FIG. 1 is a schematic diagram of a video provider and end usercommunications infrastructure in which participants comprise any numberof video 101, data 102, and/or information 103 providers, and end-userssystems 140. A provider of video, data, and/or information 101-103 isnot limited to any combination of video, data, and/or informationservices, and the services, while complementary, may be proprietary,exclusive, and independent of the services of other providers. Eachparticipant, whether principally a provider 101-103 or end user 140 isable to retrieve and transmit video, data and/or information from and toany other participant.

The delivery of video and services are herein intended to be deployableby a variety of possible communications networks, infrastructures,computer and server networks, and system configurations. FIG. 1 suggestsa plurality of networks, infrastructures, and system configurations thatmay be implemented. Shown are wired and non-wired communicationsnetworks using, for example, one or a hybrid combination of fiber optic111, coaxial cable 112, twisted copper wire 113, cellular 114, and/orsatellite 115.

A video provider, for example 101, comprises: i) communicationstechnologies 111 for establishing a plurality of video andcommunications streams to a plurality of end-users 140 to enable theuploading and/or downloading of information, data and/or video content;ii) processing hardware and software 122 for retrieving an end user'svideo preferences, content preferences, second screen functionpreferences and requests, search terms and search requests, and forprocessing the user's video preferences, content preferences, secondscreen function preferences and requests, in-video services,synchronization data, search terms and search requests. e.g., performingsearches of segment data to identify the segments or list of segmentsresponsive to a users search terms and search requests; iii) massstorage random access memory devices 123 for storing and retrievingvideo maps (e.g., segment data), and/or a videobase comprising aplurality of any combination of video segments, motion pictures,conventional programs, interactive games and services, variable contentvideos, data, in-video function services, synchronization data, and/orinformation; and iv) processing hardware and software 124 formaintaining accounting and support services in connection with video,data, and/or information services provided.

Video providers may be further categorized according to the functionsserved and/or the extent and character of the data and videobasemaintained. Video services providers 101, e.g., a cable companies, maybe capable of providing a greater variety of services than for exampleinformation providers 103, e.g., websites. Video and informationservices available over the internet are suggestive of the wide range ofmultimedia and information configurations that are possible.

A user's access to the resources of a video services provider 101-103need not be direct. A requested video may be streamed or downloaded, inreal time or non-real-time, to a services provider that may be moreeconomically accessible to the intended user. Within the network 100,some video services provider may not directly provide any services tousers, but act as centralized video originators or depositories forother services providers.

In one of many possible embodiments, an end-user video system 140acquires access to the network 100 and the various services providers101-103 via a communications device 131, e.g., cable distribution box,satellite dish. An end-users video system 140 comprises a great varietyof communications devices, computing devices, and screens 141-149.Principally communications devices include, for example, a modem 141,e.g., cable modem; an internal communications device 142, e.g., wiredand wireless router; and a network/wireless extender 143. The end-user'scommunication interfaces such as Wi-Fi, Ethernet, cellular, 4G LTE,HDMI, Optical, G.hn, and USB facilitate communications among theend-users various computing devices and multi-screen combinations144-149, which include, for example, set top box 144, e.g., cable box;PC/monitor 145; tablets 146-147; smartphone 148; and television 149. Adevice may be generally categorized as principally a communicationsdevice, a computing device, or a screen. However, devices, such astablets 146-147, smartphones 148, and portable/notebook computers 145,comprise all three functions. Further, a television screen 149 mayinclude computing, storage, and communications capabilities that mayotherwise be separately provided in a set-top box or television mediaaccessory 144.

Communications between devices may be established by any of a variety ofwired or wireless communications networks including, for example, Wi-Fiand cellular (e.g., 4G LTE) communications networks. Thus, a computingdevice need not be directly or indirectly connected by wire to a screen149. For example, a computing device 145 may be connected to a secondscreen 149 via a communications port 143. The communications port may beof varying degrees of intelligence and capabilities, it may serve toboost or manage the signal, or have no other purpose than to serve as aconvenient outlet in which to plug and unplug devices.

The particular location of an end-users devices, screens, subsystems, orcomponents, whether within the immediate boundaries of a residence orthe particular location, are not limited herein to any particulararrangement. A variety of configurations are possible to meet thevarious needs at any particular time of the end user. In a preferredembodiment an end-user configuration comprises a primary display device149, one or multiple secondary display devices such as a personalcomputer or portable computer 145, tablets 146-147, and/or smartphone148.

An application software appropriate for the communicationsinfrastructure may reside directly or indirectly in the primary displaydevice, secondary display device, and/or separate device incommunication with the primary display device and secondary displaydevice.

Multi-screen combinations include, for example, television 149 andsmartphone 148, pc/laptop 145 and smartphone 148, television 149 andpc/laptop 145, and television 149 and multiple tablets 146-147.Multi-screen combinations need not be limited to a primary screen and asecondary screen. For example, a second screen such as a tablet 146 mayprovide a second screen experience with respect to a primary screen sucha television 149, and with respect to another second screen such as asecond tablet 147.

Multi-screen usage modes may be broadly categorized as disruptive (e.g.,multi-tasking unrelated content) or complementary; sequential (e.g.,usage of a primary screen is followed by usage of a second screen) orsimultaneous; planned (e.g., expected usage of a second screen as partof the viewing of the content on a primary screen) or spontaneous;and/or dependent (e.g., highly integrated with content on the primaryscreen) or independent. However, a multi-screen usage may be, forexample, both disruptive and complementary. Disruptive, for example, inthe sense that the linear video experience is interrupted. while at thesame time complementary in the sense that information is provided thatthe user would deem advantageous in enhancing the video experience. In apreferred embodiment a second screen's user interface providesinteractive capabilities that are highly tailored and synergisticallyintegrated with a specific content displayed in a primary screen.

Implementation of the novel features detailed herein are not confined tothe communications infrastructure, services providers, and end-usermulti-screen systems detailed with respect to FIG. 1. A great manyalternate or complementary devices, systems, components, elements, andservices may be integrated in a multi-screen configuration as is, forexample, disclosed by: U.S. Pat. No. 8,516,533, titled “Second ScreenMethods And Arrangements”; U.S. patent publication 20120151509, titled“System, Method And Device For Providing A Mobile Application AcrossSmartphone Platforms To Enable Consumer Connectivity And Control OfMedia”; U.S. patent publication 20120210349, titled “Multiple-ScreenInteractive Screen Architecture”; U.S. patent publication 20130061267,titled “Method And System For Using A Second Screen Device ForInteracting With A Set Top Box To Enhance A User Experience”; U.S.patent publication 20130111514, titled “Second Screen InteractivePlatform”. Each of these cited references provide disclosures withrespect to their respective FIG. 1 that are directly related to thedisclosure above with respect to FIG. 1 and which are incorporatedherein by reference.

Video Map

The Video Map ¶å0175-0196 disclosures in U.S. patent publication20130343721, titled “Playing A Video Presentation With PlaybackFunctions” (the “'721 publication”) are incorporated herein byreference. Further, the '721 publication's disclosures with respect toPresentation Features ¶å0075-0092, and Playback Functions ¶å0093-0096are also incorporated herein by reference. With respect to a creation ofa video map, the disclosures in ¶å0117-0123 of U.S. patent publication20130251337, titled “Providing Item Information During Video Playing”are incorporated herein by reference. Further, the Video Map¶å0063-0076, Bookmark Generating Data ¶å0105-0114, Seek/Step Data¶å0115-0148, and Map Synchronization ¶å0149-0188 disclosures of U.S.patent publication 20140219630 titled “Video Map Responsive To A VideoRelease” are incorporated herein by reference. U.S. Pat. No. 6,889,383,titled “Delivery of navigation data for playback of audio and videocontent”, U.S. Pat. No. 7,975,021 titled “Method and user interface fordownloading audio and video content filters to a media player”, and U.S.Pat. No. 8,117,282 titled “Media player configured to receive playbackfilters from alternative storage mediums”, are incorporated herein byreference in their entirety.

An embodiment of such a disclosed video map and playback capabilities isimplemented in a currently freely available CustomPlay PC applicationwhich provides users a comprehensive set of video playback features andin-video functions for movies released on DVD. The CustomPlay's fourteenfeature sets include Presentations, Compilations, Subjects, Dilemmas,Best Of, Performers, Filmmaking, Plot Info, Shopping, Music, Locations,Search, Preview, and Control. The CustomPlay's eight in-video andplayback functions include Plot Info, Locations, Filmmaking, Shopping,Music, Who, What, and Play From. In the movie Casino Royale, examples ofthe video map enabled CustomPlay capabilities include 91 Shopping items,19 Locations with links to Apple Maps, 12 entertaining Subjects (1-2minutes each), 5 story-driven Presentations (26-115 minutes each), over51,000 keywords that drive the extensive Search feature, and 14content-customization categories.

A video, video map, information and content related to the video, playroutines specific to the video, and control codes for automaticallyconfiguring or controlling the functions of the end-user devices andvideo playback system may be provided by means of a variety of existingand evolving technologies. In addition to the hard formats such asoptical disc, memory chips and modules; a video may be provided by softformats, such as streaming or downloading over a network implementingany variety of video and data transmission technologies. A device neednot be physically accessible by a user or be physically located near adisplay device or screen. The device may provide a user access to remotevideo resources and may itself be remotely controlled by the user. It iswithin a network-based implementation, that the various advantages andcapabilities of the preferred embodiments are realized. The 721publication's Video Provider ¶å0250-0256 disclosures are incorporatedherein by reference.

A video map may be provided with, or separate from, the video's videoand audio data. For example, a movie may be retrieved, e.g., downloadedor streamed, from a remote video provider, and a corresponding video mapmay be retrieved from a secondary remote source, e.g., downloaded bymeans of the communications interface from a remote server. For example,a multi-screen configuration comprising processing, a memory device, andcommunications capabilities may provide playback features and in-videofunctions for a movie streamed from a remote video provider. In thisembodiment, a video map, user interface and other control programsparticular to the motion picture may be downloaded from a remote mapserver or other player. Upon a playing of a video, the control programcauses the reading of the video's identifier from the video source,searches the mass memory fixed storage device for a video map and if notavailable communicates with an external source to download theappropriate map. In this fashion conventional linear video programsprovide a library of motion pictures to which the teachings herein maybe applied.

A video map and/or a video maps' components (e.g., acoustic signaturedata) may be downloaded prior to playing the corresponding video,contemporaneously as the video is playing, and following the playing ofthe video. Some components may be downloaded prior to, while others maybe downloaded as needed, and some may be downloaded at the conclusion ofthe playing of the video. Further, the downloading of information andcontent may be responsive to a user's preestablished orcontemporaneously established features and functions preferences, theparticular multi-screen environment and second screen embodiment.

Devices

The '721 publication discloses a Multimedia Player that, in one of manypossible embodiments, may be central to the processing andcommunications in a multi-screen configuration. The '721 publication'sMultimedia Player ¶å0197-0223 disclosure is incorporated herein byreference. Still further, the teachings of currently available end-userdevices, such as plug and play boxes (e.g., Apple TV 2^(nd) generation,Roku Generation 3, Amazon Fire TV), HDMI sticks (e.g., Roku StreamingHDMI stick, Google Chromecast HDMI stick)

Specifically, with respect to second screen devices, the teachings ofsecond screen capable device such as, for example, smartphones (e.g.,iPhone, Samsung Galaxy, and Nokia Lumia), tablets (e.g., iPad Air,Amazon Kindle Fire, Microsoft Surface, Samsung Galaxy Tabs, SonyXperia), and smart TVs (e.g., Samsung UHD Series, Sony KDL Smart TVSeries) are incorporated herein by reference.

To the extent necessary, these devices may be enhanced with the additionof processing, firmware, memory, communications, and/or softwareroutines and applications to accommodate the second screen capabilitiesdisclosed herein. A processor is herein presumed to comprise a hardwareprocessor (e.g., central processing unit, CPU, and associated computingor calculating devices (e.g., graphic chips) and application software.Further, a second screen device may provide, responsive to themulti-screen configuration and user's preferences, tactile feedback(e.g., device vibration, mechanical movement of a device's key),auditory, and/or visual notification that alerts the interested viewerto the forthcoming depiction of content and functions. Tactile feedbackmay include, for example, device vibration and/or mechanical movement ofa device's key or button. A notification associated with a function orspecific function may be executed by the number of vibrations orparticular rings tones.

Remote Control Functions

A user's control of a device may be direct, e.g., key controls or screenbuttons directly on the device, or indirect, e.g., remote controls. Auser's control of a device may implement any number of technologiesincluding, for example, conventional remote controls, alphanumerickeyboard and touchpad, voice control, gesture recognition, and/or asecondary device such as a tablet or phone. Communications for a remotecontrol include, for example, infrared, Wi-Fi, and Bluetooth. The '721publication's disclosures with respect to Playback Function Controls¶å0146-0150 is incorporated herein by reference. With respect togenerating a dynamic user interface on a second screen control device,U.S. patent publication 20130007793 titled “Primary Screen View ControlThrough Kinetic UI Framework” is incorporated herein by reference.

A significant function of a second screen is to provide video playbackcontrols in addition to facilitating the display of content thatcomplements the viewing of content on primary screen. Advantageously, asmartphone and a tablet application, for example, provides all thefunctionality of the remote control and on-screen interfaces. Suchdevices may communicate with the player via Wi-Fi capability via atransmitting application in the controller and a listener application inthe controlled device.

FIG. 2 is an illustration of a smartphone remote control interface 200.Touching, a control, for example, activates the function routinesassociated with the button/object. The interface 200 comprises, forexample, interface navigation controls 201-209 (e.g., Exit 201, Settings202, Help 203, Lists 205, In-Video 206, Search 207, Browser 208, andSet-up 209); selection functions 211-213 (e.g., Up 211, Select 212, Down212); audio volume controls 215-217 (e.g., increase volume 215, decreasevolume 216, mute 217); playback function controls 221-228 (e.g.,Play/Pause Toggle 221, Play From 222, Skip Forward Clip/Scene 223, FastForward 224, Fast Rewind 225, Skip Back Clip/Scene 226, What 227, andPlay Current Clip/Scene 228); in-video function controls 241-247 (e.g.,Who 241, Shopping 242, Music 243, Locations 244, Filmmaking 245, PlotInfo 246, and Dilemma 247); and content controls 251 (e.g., contentcategory Violence and levels of explicitness None, Mild, Explicit, andGraphic).

The Exit control 201 provides confirmation to exit the application, TheSettings function 202 displays a screen that provides customization forthe display of in-video notifications on the primary screen and/or asecond screen. The Help function 203 provides context sensitive helpinformation. The List function 205 displays, in the primary screen orthe secondary screen, a menu of CustomPlay features. The In-Videofunction 206 displays a screen that provides the various in-videofunctions and play control functions shown in FIG. 2. The Searchfunction 207 displays a screen that provides Search functionality. TheBrowser function 208 displays a screen that provides mouse and keyboardfunctionality. The Set-up function 209 displays a screen that providesaccess to the various utilities for establishing communications withother devices (e.g., IP address and port number).

The Play From control 222 enables a user to continue playback from acurrent position in the video irrespective of the particular featurethat is currently being used. For example, a user may utilize the Searchfeature to obtain a list of segments that are responsive to a keywordsearch. While playing one of the segments in the list, the user mayactivate the Play From control to play the video from that point in thesegment. Advantageously, the Play From control is further responsive toa user preestablished presentation preference (e.g., Play As Is orCustom presentations). Without a preestablished presentation preferencethe Play From control may be set to default to the Custom presentationrather than, for example, the Play As Is presentation. Alternatively,the Play From control defaults to playing the last presentation playedby that user for that movie, or, a presentation may be selectedresponsive to the last feature utilized. For example, and responsive toa user preference, if the Search or Preview feature is last utilized,the Play From control enables the Play As Is presentation. If on theother hand, the Best Of feature is last utilized, the Play From controlenables the Custom presentation.

The What replay control 227, when enabled by the user during playback,rewinds the video a user defined amount of time, and replays a portionof the video with the subtitles enabled. With respect to replaycapabilities, the disclosures of U.S. Pat. No. 6,408,128 titled“Replaying with supplementary information a segment of a video”; andU.S. Pat. No. 7,430,360 titled “Replaying A Video Segment With ChangedAudio”; are incorporated herein by reference.

The Play Current Clip/Scene control 228 utilizes the clip and scenedatabase portions of a video map to identify a clip or scene definitionthat is responsive to the current play position and automatically rewindto the beginning of the clip or scene and plays from that position.Whether a Clip/Scene control (Skip Forward Clip/Scene 223, Skip BackClip/Scene 226, Play Current Clip/Scene 228) is responsive to a Clip orScene is user a predefined option.

Second Screen Functions

Generally, a second screen function is a type of in-video function thatis enabled on a second screen or a multi-screen system during videoplayback on a primary screen. With respect to second screen apparatus,systems, architectures, methods, and functionalities, the followingdisclosures are incorporated herein by reference: U.S. Pat. No.8,516,533 titled “Second Screen Methods And Arrangements”; U.S. Pat. No.8,424,037 titled “Apparatus, Systems And Methods For Accessing AndSynchronizing Presentation Of Media Content And Supplemental Media RichContent In Response To Selection Of A Presented Object”; U.S. Pat. No.7,899,915 titled “Method And Apparatus For Browsing Using MultipleCoordinated Device Sets”; U.S. patent application 20120210349 titled“Multiple-Screen Interactive Screen Architecture”; U.S. patentapplication 20130061267 titled “Method And System For Using A SecondScreen Device For Interacting With A Set Top Box To Enhance A UserExperience”; U.S. patent application 20130014155 titled “System AndMethod For Presenting Content With Time Based Metadata”; and U.S. patentapplication 20140165112 titled “Launching A Second-Screen ApplicationRelated To A Non-Triggered First-Screen Application”.

The teachings of currently available second screen capabilities andfunctions, such as Google's Play Info Cards, Amazon's X-Ray, Xbox'sSmartGlass, iTunes Extras, X-Men Movies Cerebro App, IMDb App ForiPhone, iPad & iPod Touch, and RunPee, are incorporated herein byreference. A number of methodologies are available to synchronize manydifferent devices through Wi-Fi networks and/or remote servers (e.g.,JargonTalk).

A video map that enables the CustomPlay's fourteen feature sets andeight in-video and playback functions provides the information necessaryto enable the variety of synergistically integrated multi-screencapabilities. The '721 publication's disclosures with respect to thePresentation Features ¶å0075-0092, Playback Functions ¶å0093-0096, WhereFunction ¶å0097-0122, Why Function ¶å0123-0129, How Function¶å0130-0139, and Info Function ¶å0140-0145 are incorporated herein byreference and implemented as complementary or alternative second screenembodiments. The Where Function, Why Function, and How Function arereferred to herein, respectively, as the Locations Function, Plot InfoFunction, and Filmmaking Function.

It is intended by this specification that the features, playbackfeatures, playback functions, in-video functions, and functionsdisclosed herein and incorporated by reference may be implemented invarious second screen embodiments that do not require altering aconventional playing of a video on a primary screen (e.g., remotecontrol functions on the second screen, superimposing notificationindications, seamless skipping of video segments, and selective playingof segments).

Second screen functions may take advantage of any additional videocontent (e.g., video/audio commentary, or supplementary video/audiocontent) that is supplied with or is available for the movie. In thecase of additional video content that is supplied with the movie, thevideo map would, advantageously, map the video/audio segments fromwithin the additional video content, and associate descriptors includingsynchronization data (e.g., references or linkages). A second screenfunction would to utilize the synchronization data to provide additionalinformation (e.g. additional video content) during playback of themovie.

What Second Screen Function

FIG. 3A is an illustration of a second screen display of features of aWhat function. The illustration of a user interface 301 represents aclip in the motion picture Casino Royal in which 007 approaches an AstonMartin automobile. The clip is followed by a clip depicting 007 seatedin the car and delivering a line in response to opening an envelope. Inthis example, the user desiring to understand what he said activates theWhat function control 302 which causes the display of the subtitles 303of the recently delivered line: “I love you too, M”. The What functioncontrol 302, optionally and/or responsive to the particular secondscreen embodiment, may not rewind the video. Instead, when enabled bythe user, and responsive to the user preference, the activation of theWhat function 302 displays the subtitles 303 only on the second screen301 for a portion of the video recently played. The video itself is notreplayed. Unlike the disclosure of the previously incorporated U.S. Pat.No. 6,408,128, this function enables one user to display the subtitles303 for a not understood dialogue portion without interfering with theviewing experience of a second user. Since multitasking of features andaccessing additional external information is an advantage of the secondscreen application, the chances are increased that a user may missdialogue the user would not otherwise have missed.

An activation of the What function (e.g., touching the What functionicon 302 on the second screen interface 301) causes the second screen'sprocessing to identify the current play position either directly orindirectly from a playing of the video on a primary screen and/or fromthe second screen's internal play clock synchronized to the videoplayback. This request position is utilized to define a period of timewithin the video for which subtitles will be displayed. A system defaultand/or a viewer's preestablished preferences determines: the period thatsubtitles are displayed (e.g., “replayed” 20 seconds); the subtitlelanguage (e.g., English or another user preferred language) depending onthe availability of sub-title information; the type of the subtitles(e.g., subtitles, closed captioning, descriptive content, andcommentary); and the time to discontinue displaying subtitles (e.g., ateither the point the What function was activated or at some systemspecified and/or a viewer preestablished preferences with respect to thetime at which the What function was activated (e.g., five seconds priorto, or after, the point the What function was activated). The video mapsubtitle data is searched to identify the subtitle informationcorresponding to the desired period of time. The appropriate subtitlesare then displayed, responsive to the particular second screenembodiment, on the primary screen and/or the second screen, withoutrequiring a rewinding and replay of the video.

By contrast to the methodology implemented when the video is replayedwith subtitles, in the case of a second screen embodiment there are anumber of novel methodologies that may be implemented depending on, forexample: the nature of the primary screen/video playback device andsecond screen communication; whether a separate device/software isplaying the video and whether it is capable of multitasking anuninterrupted playing of the video while retrieving the necessarysubtitle information; and/or whether the subtitle information may beretrieved by the second screen independently of the video playbackcapabilities and inclusion of subtitles with the video. In a preferredstand alone second screen embodiment, the subtitle information isdownloaded from a local or remote memory device/server. In oneembodiment, as part of a downloading of a video map, the subtitleinformation is downloaded together with subtitle synchronizinginformation. The synchronizing information enables the second screen tosynchronize the display of subtitles to the video playback on theprimary screen. In this example, the subtitle information andsynchronizing data are provided by a third party or parties. Since thevideo is not being replayed, such a display of subtitles need not beexactly synchronized with the audio track, a delay or offset may beadvantageously implemented prior to delivering the data to the secondscreen. Thus, the synchronizing information may already offset thedisplay of subtitles.

Alternatively, to an intermittent activation of the What function, aparticularly inventive continuous display of subtitles implements acounterintuitive advantageous offset synchronization methodology. Thisimplementation is innovatively distinct from the display of closedcaptioning on a primary screen principally because a haphazard delay inthe display of closed captioning is not advantageous and is onlyavailable on the primary screen.

A display of subtitles may be responsive to a user preestablishedpreference with respect to all videos, a specific category of videos(e.g., Foreign Films), specific sequences within a video (e.g., sceneswith Heavily Accented dialogue, or Non U.S. Accented English dialogue),and specific portions comprising dialogue that is deemed unclear. If theactivation of the What function has not been preestablished, then a usermay activate, at any time, the What function in continuous mode bypressing and holding the What function control 302. This will enable theWhat function in continuous mode rather than the temporary mode that isenabled by only touching the What function control 302. The continuousmode may be deactivated by a subsequent touching of the What functioncontrol 302.

If the What function has been activated in continuous mode then thesubtitles or preselected portions of the subtitles are displayedcontinuously in a synchronized offset fashion. In this context, theconcepts of the terms offset, delay, and related synonyms, are similarlyapplicable and interchangeable. FIG. 3B is an illustration of offsetsynchronization methodologies. Conventionally, subtitles are displayedas close as possible to the actual dialogue in the audio track. FIG. 3Billustrates a subtitle unit “I love you too, M,” 322 synchronized inconventional manner with an audio track including a dialogue portion321. FIG. 3B also illustrates an offset synchronization of the displayof subtitles that may be controlled by one or a combinations of offsetmethodologies, including, for example: the subtitleportion/sentence/phrase (subtitle unit) is displayed the equivalent of asubtitle unit(s) 323 after the corresponding accurately synchronizedsubtitle would have been displayed 322; the subtitle is displayed at apreestablished play position 324 after the position at which thecorresponding accurately synchronized subtitle would have been began tobe displayed 322; and the subtitle is displayed a system default and/ora viewer's preestablished time offset period 325 (e.g., a delay of 2seconds) after the time at which the corresponding accuratelysynchronized subtitle would have been displayed or began to bedisplayed. Contrary to the haphazard unsynchronized delay between areal-time created closed captioning and the corresponding dialogue, in aWhat embodiment, a subtitle offset synchronization is controlled.

A continuous mode does not necessarily mean that all of the subtitleunits are played. Continuous mode may only play, for example, a subsetof the subtitles (e.g., 10-12 subtitle segments corresponding to thoseselect instances in the video that the audio was least clear and/or thebest lines in the video). An intermitting continuous mode willmaterially reduce the amount of subtitles that are displayed and thusreduce the subtitle data needed to be provided to the second screen.

Exemplary embodiments comprise: downloading, over a communicationsnetwork from a remote information provider, supplementary information(e.g., subtitles) to a memory of a second screen device; receivingsynchronizing information that is responsive to a playing of a video ona primary screen device (e.g., the second screen device receiving, overa Wi-Fi network, from the primary screen device an identification of acurrent play position of a video being played on the primary screendevice; or using a built-in microphone to receive audio information,principally from a video playing on a primary screen device, to generatean acoustic fingerprint; or downloading synchronizing information from aremote information provider); establishing, responsive to thesynchronizing information, a synchronization to the playing of the videoon the primary screen device (e.g., utilizing the identification of acurrent play position or comparing the acoustic fingerprint to anacoustic database to determine the current play position of the videoplaying on the primary screen device); and displaying the supplementaryinformation on the second screen device responsive to an offsetsynchronization to the video playing on the primary screen device,whereby the display of supplementary information on the second screendevice lags a contemporaneous playing of the video on the primary screendevice.

Individual steps outlined above may be performed by the primary screendevice, the second screen device, a remote service provider, alone or inlogical combination. For example, an identification of a current playposition may be performed by the primary screen device, the secondscreen device by means of generating and comparing an acousticfingerprint, or by a remote service provider by means of analyzing audioinformation received from the second screen device over a computercommunications network. When remote processing is involved, the offsetsynchronization may be performed by the remote processing, and thedownloaded supplementary information may be adjusted accordingly. Offsetsynchronization methodologies may be advantageously applied to otherfunctions, display of information and display of supplementaryinformation embodiments disclosed herein. Different synchronizationmethodologies may be simultaneously active within a single display, eachresponsive to, for example, the relationship of the supplementaryinformation to the depiction presently played on a primary screen.Further, advantageously, a user is provided the option to deactivate,activate, select the particular offset synchronization methodology,and/or adjust the offset delay parameters (e.g., time delay and subtitleunits delays). Deactivating the offset in synchronization restores aconventional synchronization.

Notwithstanding the conceptual distinctions, the previously incorporatedU.S. Pat. No. 6,408,128 discloses at 46:57-53:19 a number of routines,elements, and features that may be synergistically integrated with theinventive implementations and embodiments disclosed herein. For example,similar to the disclosed rewinding of the video being cumulativeresponsive to multiple successive activation of the What function, in asecond screen embodiment, the period of time that defines the display ofsubtitles is also cumulative responsive to multiple successiveactivation of the What function.

ID and Credits Second Screen Functions.

The user interface 301 of FIG. 3A also comprises an ID function control305 and a Credits function control 311. A user desiring to obtainidentification information activates the ID function control 305 whichcauses the display of the identification of the noteworthy item/action.The identification of an item comprises, for example, an image from thevideo and/or a promotional image 306, written identification 307 (e.g.,brand name, model, description “Aston Martin DBS Sports Car”, type ofaction and name of the action “martial arts butterfly kick”), awrite-up, and access to additional information 308, and/or linkages toexternal sources providing further information about the item/action(e.g., Wikipedia page) 309.

Further, the ID function may synergistically take advantage of the dataassociated with a Subject presentation. For example, the Subject “TheDude” in the movie The Big Lebowski, the subject identifies over 100instances of the dialogue including the term “Dude”. During an ID Gamefunction, a notification is displayed for each instance in theparticular ID Game set to enable the participants to take the particularaction that they have associated with the game (e.g., deliver a bestline from the movie). An ID Game notification may also include thepresentation of Trivia function information/question, and/or thepresentation of a simplified Yes/No question, to determine if theparticular action that has been associated with the game or theparticular notification instance is to be performed.

A primary objective of the Credits function is to inform the user thatthere is additional content during and/or after the playing of thecredits. A user may query the function at any time by pressing theCredits function control 311 which enables routines that cause thedisplay of an appropriate message. In this example, the user is informedthat there is no content during or after the credits 312.

By contrast, in the motion picture The Avengers after one minute andfifty six seconds of the films credits, a short scene depicts the evilsuper villain Thanos talking to his emissary The Other is played. Thecredits resume for six minutes and seventeen seconds, and a final scenedepicting the Avenges eating at a Middle Eastern restaurant concludesthe credit sequence. In this example, activating the Credits functioncontrol 311 causes a display informing the user that: “There are twoseparate sequences during the credits.”

The Credits function may also, responsive to system default or userpreference, be automatically activated immediately as the credits beginto be displayed on the primary screen. Such an automatic activationwould inform the user on the primary and/or secondary screen of whetheror not there is additional content that follows. Additionally,responsive to a user's preference or the activation of the Creditsfunction control, the playing of the movies is automatically terminatedif there is no content during or after the credits. If there is contentduring or after the credits, responsive to the video map, thenon-content portions of the credits are seamlessly skipped. In otherwords, with respect to the motion picture The Avengers, the first minuteand fifty six seconds of the films credits are skipped, a short scenedepicting the evil super villain Thanos talking to his emissary TheOther is played, the following six minutes and seventeen seconds ofcredits are skipped, and the final scene depicting the Avengers isplayed.

As in other examples, the illustration of the icon and label for afunction control may be responsive to the object of the function. Forexample, in the case of the Credits function control 311, if there iscontent during or after the credits, the icon of a glass is depicted asnot being empty, and the label reads, for example: “There's More”.

By contrast to the Credits function that enables a user to see all ofthe content of a video, a Synopsys function enables a user todiscontinue viewing a video at any point and still find out how the plotof the video resolved itself. The Synopsys function mapfragments/defines the video into chapters, scenes, clips, and/or periodof time to generate multiple units of the video. Each unit is associatedwith a time code corresponding to a video's play position at which theunit begins. A short write-up synopsis (e.g., a paragraph) is createdfor every unit in the video.

When a user activates a Synopsis function control, an identification isobtained of a stop play position in the video playing on the primaryscreen device. Responsive to the identified stop play position, thesynopsis for units with a time code that follows the stop play positionare provided the user as a brief summary of the not viewed portion ofthe video. The write-up for a unit that immediately precedes stop playposition may also be included in the summary. The synopsis summary maybe displayed as a series of paragraphs preceded by the beginning timecode. Advantageously, the synopsis summary enables the user to easilyobtain information relating to the conclusion of a video which for onereason or the other they may not desire, or be able, to finish viewing(e.g. airplane landing).

An activation of the Synopsys function control may also provide the usera number of options. First, a user may be asked to confirm the displayof the synopsis summary by requiring a reply to a confirmation message(e.g., “Would you like to know what happens in the film from this pointforward?”) Second, and advantageously, in those Features capableimplementations, the user is provided to transfer to a 60 Minutespresentation beginning at a point in the presentation that is responsiveto the stop play position. Further, instead of a written write-upsummarizing individual units, a specifically prepared video synopsis mayinstead be provided. In such a case, the video synopsis summary may be ageneralized single piece that is independent of a stop play position,rather than fragmented units which are combined in real-time responsiveto the stop play position and which would play as a single videosynopsis summary.

Who Second Screen Function

The Who function identifies the performers and characters being depictedand provides access to resources, e.g., websites, for additionalinformation. With respect to the Who function, the disclosure of U.S.Pat. No. 8,494,346 titled “Identifying A Performer During A Playing Of AVideo” is incorporated herein by reference.

FIG. 3C is an illustration of a second screen display of features of aWho function. Responsive to preestablished user preference and thecurrent play position, Who information is displayed on the second screen331 contemporaneously with the playing of the video on the primaryscreen. The contemporaneous display may be in strict synchronization(e.g., only when the character is depicted on the screen), implement anoffset synchronization, and/or implement a segment responsivesynchronization (e.g. for the entirety of a clip in which a character isdepicted for at least some portion). In this exemplary illustration, auser had elected to activate the Who function control 332 to executecontemporaneously as the video is being played. Thus, responsive to thecurrent play position, the depicted noteworthy performers/characters areidentified (e.g., David Hope 341, Jenn Starlight 342, and Jonny Deep343).

The Who data comprises, for example, a visual depiction of the performerof a character and the performer's name. With respect to movie andperformer information and features available in a second screenapplication, the teachings of the IMDb App for iPhone, iPad & iPod TouchVersion 4.4 is incorporated herein by reference. Additionally, the Whodata comprises a character's name and an image of the character. In thisillustration, the image of a character is extracted from within thevideo and displayed 351, together with the performer's name (“JennStarlight”) 352, and the character's name (“Sunshine”) 353.

Each character is also associated with noteworthy features and/orin-video functions extracted from and related to the movie. For example,David Hope 341 is associated with a Subject featuring the character'sparticular mannerisms; and Jenn Starlight 342 is associated withShopping items that she wears in the movie. Pressing the suggestedfunction control 354 launches the appropriate function either on theprimary screen and/or the secondary screen. In the case of the JennStarlight 342 illustration, the Shopping function control 354 launches awebsite where shopping information about the displayed stripe dress isprovided.

The interface, taking advantage of the Search feature of a second screenapplication, also provides a user the capability to automatically search355 within the video for instances of the same currently highlightedperformer 342. For example, a user may access the best lines of acharacter, close-up presentation, and product placements featuring thecharacter. Similarly, a user is provided the means (e.g., Internet iconcontrol 356) to obtain external information relating to the performer(e.g., filmography website page), and the means (e.g., SUB icon control357) to play a presentation of the plurality of video segments (e.g., aSubject) within the video relating to the currently highlightedcharacter/performer 342. This exemplary embodiment also comprises thepreviously detailed What function control 358 and the display ofsubtitles 359.

Locations Second Screen Function

The Locations function identifies the locale being depicted and providesaccess via linkages to relevant locale information and geographicalmaps. The '721 publication's disclosures with respect to the WhereFunction ¶å0097-0122 are incorporated herein by reference.

Data created for, or utilized by the Locations function comprises, forexample: a segment definition; an identity, as represented in the video,of a locale depicted within a video segment defined by the selectedsegment definition; an actual identity of the locale depicted within thevideo segment; a linkage, responsive to the actual identity, to ageographic map; a video frame identifier to provide a reference imageand thumbnail; a write-up to provide brief in-video information; and alinkage to more detailed information. For example, in the motion pictureThe Dark Knight, the video map defines a segment by frames numbers8797-8990 (23.976 frames per second) for a segment including thedepiction of the exterior of the building represented in the movie to bethe Gotham National Bank. The video map provides a locale identity asrepresented in the video (e.g., Gotham National Bank), and an actualidentity of the locale (e.g., Old Chicago Main Post Office). The videomap for the motion picture The Dark Knight also includes anidentification of video frame number, i.e., 8989, that is associatedwith the segment definition to provide a reference image and thumbnailfor the locale.

Responsive to user's preferences and the particular multi-screenembodiment, an indication that information is available is provided onthe primary screen (e.g., superimposed over video frames or over theunused portion of the screen, letterbox or curtains (i.e., notsuperimposed over the video frame). FIG. 4A is an illustration of asecond screen display of features of a Locations function 400. In asecond screen embodiment, a user upon receiving indications thatinformation is available, e.g., a Locations icon is displayed on theprimary screen, and/or the Locations function control is highlighted401, the user may elect to enable a display of Locations information byactivating the Locations function control 401.

In the illustration of FIG. 4A, the Locations information comprises animage of the locale 411, a globe icon 412, an identification of thedepicted locale 413, and an identification of the actual locale 414. Theimage of the locale 411 helps distinguish among multiple locales thatmay be identified at this location, clip or period. Additionally, theimage of the locale 411 enables the user to directly access, forexample, a SUB presentation of the locale, a plurality of video segmentsfrom within the video depicting the locale, or a write-up 415.Preferably, in a second screen embodiment, the write-up 415 is providedwith the other particulars of the identified location.

Optionally, as illustrated, the image of the locale 411 is focused onthe portion of the video frame depicting the target locale.Alternatively, the image of the locale 411 is a video presentation ofthe locale rather than a single image. The globe icon 412 enables a userto automatically retrieve a geographical map locating the locale. Eachof the identifications 413 414 enable the user to automaticallyretrieve, for example, corresponding website pages where additionalinformation is available.

In the example of Locations function, a user may request the display ofa geographical map locating the locale. To that purpose, a video map,associates, for example, a Google map link with a video location (e.g.,a segment definition).https://www.google.com/maps/@41.8767574,−87.6394313,3a,75y,102.34h,93.34t/data=!3m4!1e1!3m2! 1sTICI1ltiA5J6fii4e HapoQ!2e0 isan example of Google map link for a locale at 433 West Van Buren Street,Chicago, Ill. 60607 showing the Old Chicago Main Post Office depicted inthe motion picture The Dark Knight. The Google map link includes asubstantial amount of data (e.g., parameters). For example, a URL thatstarts with “www.google.com/maps/@” and with a “1e1” second parameteridentifies a Street View, with a “1e2” second parameter it identifies aPanoramio, and with a “1e3” second parameter it identifies a Simple Map.Latitude and longitude are identifies by the data that immediatelyfollows www.google.com/maps/@ (e.g., 41.8767574,−87.6394313). Similarly,the link's data provides a heading (e.g., 102.34h), and a pitch (e.g.,93.34t).

FIG. 4B is an illustration of a second screen display of a set ofgeographical maps that are displayed responsive to a user activating ageographical map display function (e.g., activating the Locationsnotification icon, the Locations control function, or the globe icon),or responsive to a preestablished preference for a continuous activationof the Locations function. The second screen display 441 provides threeseparate map views generated by parsing the data provided by the aboveGoogle map link, inserting the desired parameters into HTML templates,and utilizing the Google Maps Javascript API. The map on the top of thescreen 442 is a Street View. The map on the lower left 443 of the screenis the same latitude and longitude as the Street View but displayed as amap, not as a Street View. The map in the bottom right hand 444 of thescreen is the same latitude and longitude but 6 levels less of zoom ifthe original zoom was smaller than 16. If the original zoom level was 16or more, then it uses an arbitrary zoom level of 10. A similar method isused in the case of a Panoramio. As is depicted in FIG. 4B, each of theviews provides a different perspective and provides individual separatenavigation 451, e.g., movement and magnification.

Exemplary embodiments comprise: receiving, on a second screen device,synchronizing information that is responsive to a playing of the videoon a primary screen device; establishing, responsive to thesynchronizing information, a synchronization to the playing of thevideo; retrieving (downloading, over a communications network and to amemory of a second screen device, and/or retrieving from a memory of asecond screen device) a linkage to a geographic map, a write-up, and anactual identity of a locale, each corresponding to a locale depicted inthe playing of the video; parsing the linkage to the geographic map toenable a display of multiple geographic map views each corresponding tothe locale depicted in the video, each of the geographic map viewsproviding a different perspective and individual navigation; andsimultaneously displaying the multiple geographic map views, thewrite-up, and the actual identity of the locale, on the second screendevice responsive to a time period offset synchronization to the videoplaying on the primary screen device, the time period offsetsynchronization being responsive to a user's time period offsetpreference, whereby the display of multiple geographic map views on thesecond screen device lags a playing, on the primary screen device, ofthe corresponding locale depicted in the video.

Alternatively, or in addition, to the parsing of a single link, multiplelinks corresponding to a single locale depicted in a video may bedownloaded to the second screen device and/or retrieved from the secondscreen device to access single/multiple map views and/or relatedinformation from a single or multiple sources of map views and relatedinformation. Generally, a displaying of supplementary information (e.g.,multiple geographic map views) on the second screen device responsive toa synchronization to the video playing on the primary screen device doesnot by itself establish the precise point at which the supplementaryinformation begins to be displayed or is discontinued form beingdisplayed. In other words, a displaying of supplementary information maybe responsive to a synchronization to a video playing while at the sametime being responsive to another consideration (e.g., a user'spreference, the particulars of the function, and the nature of theinformation such as the capability to navigate a displayed geographicmap view). Further, a simultaneously displaying should be understood asbeing responsive to, for example, different response and download times,and occurring contemporaneously (e.g., a similar period of time). Theterm contemporaneously is not limited to the same time. For example,information being displayed contemporaneously with a depiction ofrelated content, may be displayed after the completion of the depictionof related content provided that such delay is functionallyadvantageous, or responsive to a user's preference or system default.

The particular second screen embodiment that is illustrated alsocomprises an Exit control 451, a Help control 452, a play/pause control453, and in-video functions generally available (e.g., Control 461, andWho 462). Further, the exemplary second screen advantageously includescontext sensitive controls that also serve to identify the availabilityof information that is responsive to the play location. In this example,responsive to what is depicted in the movie the context sensitivecontrols include a Shop control 471, a Locations control 472, and aFilmmaking control 473. In this interface, the Locations control 472serves to, for example, toggle the display of Locations informationbetween the map and other locale information, and to bring the maps tofull screen.

Plot Info Second Screen Function

The Plot Info function provides plot information, rationale, orexplanation. Data created for, or utilized by, the Plot Info functioncomprises, for example: a segment definition; a plot explanation for aplot point depicted within a segment of the movie; and a linkage to moredetailed information. For example, in the motion picture Inception, atotem (e.g., a spinning top) plays a key function in understanding keyprinciples in the movie. Advantageously, throughout the movie the PlotInfo function provides the user the opportunity to obtain explanationsrelating to the totem. A segment definition defining a video segmentbeginning at video frame 22707 and ending at video frame 23432, depictsa totem, and is associated with the following plot explanation: “InInception, a totem is a small object, unique to each person, that aperson must carry at all times in order to be able to distinguish dreamsfrom reality.” Viewing a motion picture with the Plot Info notificationfunction enabled is particularly beneficial in a second viewing of anintellectually challenging motion pictures like Inception and theMatrix. A video map may associate the segment definition with ageneralized additional information link (e.g.,http://en.wikipedia.org/wiki/Inception_%28movie %29) or a plot specificadditional information link (e.g.,http://dharmafolk.org/2011/01/10/whats-your-inception-totem/)

In important feature of the Plot Info function is to bring attention tocertain clues as they occur in the video. Clues are easily-missed itemsor moments that are important to obtaining a complete understanding of amovie. These may be the little things a viewer does not normally noticeuntil a second viewing of a movie. The Plot Info explanation for a cluedraws the viewer's attention to an important element without spoilingthe movie or giving away too much. For example, in the movie Inception,the “Cobb's Wedding Ring” Plot Info notification appears the first timeCobb's ring is easily visible for a long enough period of time. A PlotInfo clue notification merely draws attention to this detail without anyfurther explanation of significance. Later in the movie, when the viewerhas seen Cobb wearing his ring inside of known dreams and not wearinghis ring outside of dreams, the Plot Info function can now explain theapparent significance of the ring.

A second screen embodiment is particularly advantageous in thoseinstances that a movie's plot turns on certain clues since it enables auser to obtain clue information without impacting the viewing of othersthat may be present. To that end, a user may prefer that Plot Infonotification relating to plot information in general and/or clues inparticular are not displayed on the primary screen. Further a viewer mayprefer to obtain plot information, but would prefer not to be assistedwith recognizing clues. To that extent the video map furtherdistinguishes the type of Plot Info (e.g., plot explanation, plotsummary to this point, clue), the second screen Settings functionenables the user to preestablish which category or categories Plot Infonotification will be active and for which category or categories PlotInfo information is displayed.

FIG. 5 is an illustration of a second screen display of features of aPlot Info function. In this exemplary second screen embodiment 501,responsive to the current video play position and the video map, thedisplay of Plot Info function control 511 is highlighted. Following auser's activation of the Plot Info function, a context sensitiveon-screen plot explanation is displayed 512. Alternatively, the contextsensitive on-screen plot explanation 512 is displayed automaticallywithout the user activating the Plot Info function 511 at thatparticular point. The on-screen plot explanation 512 may remaindisplayed on the second screen 501 until a next information display, theuser toggles the Plot Info control 511, or after a predetermined periodof time.

The Plot Info function, taking advantage of the Search feature of theplayer application, also provides the user the capability toautomatically search 521 for instances of the same plot point, andobtain progressively fuller plot explanations. Similarly, takingadvantage of a SUB feature, the user is provided the means to play apresentation of the plurality of video segments within the videorelating to the particular plot point (e.g., the totem) 522. Further, alinkage may be provided to external information or discussion on thetopic 523. As illustrated in FIG. 5, in a preferred embodiment eachcategory of additional information (e.g., keyword search 521, SUBplayback 522, link to dedicated location, and external link to thirdparty websites 523) would have their own identifiers that may berecognized across the various features and functions of a second screenapplication.

Filmmaking Second Screen Function

The Filmmaking function provides cinematic technical information andexplanation on how a scene, clip, or segment was created. The '721publication's disclosures with respect to the How Function ¶å0130-0139are incorporated herein by reference and may be implemented ascomplementary or alternative second screen embodiments.

Data created for, or utilized by, the Filmmaking function comprises, forexample: a segment definition; a write-up to provide, for example, anin-video explanation of how the shot, clip, or scene was filmed/created;a linkage to more detailed technical information, and a linkage to videospecific information. Filmmaking information may also include a playableclip of the sequence from the movie, a playable video from behind thescenes, production photos and other pictorial material, playable audiotrack from the director's commentary, and/or any other information thata user of the Filmmaking function may deem informative and/orentertaining. As herein defined, a write-up is not limited to thewritten word. Specially, with the Filmmaking function, it isparticularly advantageous that the write-up be, in fact. a multimediapresentation including video and audio (e.g., a video) to better explaina particular cinematic technique. The video presentation should be briefto enhance, rather than detract, from a conventional movie playbackexperience.

The Filmmaking function is principally dedicated to cinematography asart and as technique. For example, in the motion picture Citizen Kane,the video map defines a segment by frames numbers 39903-40330. Thesegment relates to the scene when Kane's business manager and Kane'spersonal banker are reading a contract that relinquishes Kane'sinvolvement with his newspaper after it has gone bankrupt. Kane entersthe frame and slowly walks towards the back of the office. Thebackground behind Kane is a wall of windows, seemingly of normal size.As he keeps walking, the wall appears much higher and further away thanit initially seemed and Kane appears to be dwarfed by it. The write-upto support the Filmmaking function comprises the following explanation:“The director, Orson Welles, and cinematographer, Gregg Toland, used acombination of forced perspective and deep focus cinematography. Inforced perspective, an optical illusion is created to depict a subjector object to appear farther away, closer, larger, or smaller than itactually is. The set of the office was built on an enormous scale, withthe office being much larger and the walls much taller than they appear.Deep focus allows objects in the background (in this case, CharlesFoster Kane) and foreground (Mr. Bernstein and Thatcher) of a shot to besimultaneously in sharp focus.” The video map associates a forcedperspective and a deep focus wiki links to provide additional technicalinformation.

Advantageously, a Filmmaking second screen embodiment enables a user toobtain information on the second screen while at the same time the videois available for playback functions on the primary screen. For example,a user may, while reading an explanation of a clip on the second screen,rewind and analyze the movie clip on the primary screen.

The Filmmaking function may implement an interface similar to that of,for example, the Plot Info interface illustrated in FIG. 5, The displayof Filmmaking control 531 is highlighted and the context sensitiveon-screen Filmmaking information is displayed. In this instance, thecinematic information identifies the particular technique utilized(e.g., Jib Up). The Filmmaking function, taking advantage of the Searchfeature of the second screen application, also provides the user thecapability to automatically search for instances of the same filmmakingtechnique. Similarly, taking advantage of a SUB feature, the user isprovided the means to play a presentation of the plurality of videosegments within the video relating to the filmmaking technique. Further,a linkage may be provided to external information or discussion on thetopic.

Dilemmas Second Screen Function

The Dilemmas function identifies depictions of conflicts between twocompeting moral principles such as compassion, financial responsibility,obligation to employees, parental responsibility, pursuit of soul mate,and self-actualization. A Dilemma is a segment or a set of segments thatdepict a character (also referred to as the agent) facing a decision inwhich two moral principles are in conflict. A video map defines eachsegment or a set of segments that defines a Dilemma and provides adilemma title, a relationship descriptor, and two competing moralprinciple descriptors. A relationship descriptor is one of a number ofpossible descriptors that categorizes the social relationship of theagent, e.g., Spouse/Romantic Partner, Parent/Child, Family, SocialRelationships, Government, Business/Profession, and Inner Self Tofacilitate search across a plurality of movies, the relationshipdescriptors and the moral principle descriptors are standardized. Thevideo map may also comprise a more detailed identification of the moralprinciples, a generalized moral dilemma question that is subject matterrelated to the moral dilemma depicted within the video, a dilemmawrite-up to provide, for example, an explanation of the specific moraldilemma being depicted; a linkage to in-depth philosophical analysis, arating, and/or any other information that a user of the Dilemma functionmay deem informative and/or entertaining.

An example of a Dilemma is depicted in the movie Spider-Man 2 when PeterParker's secret identity as Spider-Man has finally been revealed toMary-Jane. She now knows why Peter has been so resistant to pursuing arelationship with her: if his enemies knew his real identity, it wouldendanger the people that he loves. In this example, the relationshipdescriptor is “Spouse/Romantic Partner” and the two moral principledescriptors are “Pursuit Of Soul Mate” and “Protection Of A Loved One”.

FIG. 6 is an illustration of a second screen display of features of aDilemma function. In this exemplary second screen embodiment 601,responsive to the current video play position and the video map, thedisplay of Dilemma control 611 is highlighted, and context sensitiveon-screen Dilemma information is displayed. In this illustration, theDilemma information identifies a relationship descriptor 621 (e.g.,Spouse/Romantic Partner), and a first moral principle descriptor 622(e.g., Pursuit Of Soul Mate), and a second moral principle descriptor623 (e.g., Protection Of A Loved One). The Dilemma information alsocomprises generalized moral dilemma question 625 that is subject matterrelated to the moral dilemma depicted within the video (e.g., “Would youpursue a relationship with a potential significant other that mayphysically endanger him/her?”). A specific moral dilemma question wouldask what the user would do in the situation depicted within the movie(e.g., take the place of Peter Parker/Spider-Man).

The Dilemma function enables a user to selectively search, individuallyor in combination, a relationship descriptor 631, a first moralprinciple descriptor 632, and/or a second moral principle descriptor633. To facilitate the search, drop-down lists are provided for eachfield. In this example, Boolean capabilities are included by toggling asearch box to indicate an exclude condition 631 (an X mark), the includecondition 632 (a check mark), or a do not consider condition 633 (ablank box). Activating the search control 630 would cause a list displayof the matching Dilemmas. Alternatively, since in one embodiment thedefault condition is a match, the user may automatically search forinstances of a dilemma having the same moral principles in conflict byactivating the search control 630.

The Dilemmas function is advantageously designed to stimulatediscussions among viewers regarding the moral dilemmas depicted within amovie. It provides an opportunity to compare and contrast the decisionsviewers would make under similar circumstances, and to stimulateintellectual discussions that reveal an individual's character andvalues. A moral dilemma presentation can serve to stimulate familyconversations that could lead to a child's moral development andcharacter growth. Within a classroom, teachers may use moral dilemmapresentations to showcase ethical practices and moral principlesrelevant to their subject matter.

A Dilemma second screen embodiment, responsive to a user's preferences,is configured to facilitate any of a number of educational, informative,and/or entertaining activities. For example, in one second screenembodiment, one individual (e.g., a parent/teacher) has access to therelationship descriptor and the moral principle descriptors and is ableto query others (e.g., children/students) to identify them.

In an alternate embodiment, immediately after a Dilemma has beenpresented during a playing of the video on a primary screen and beforethe outcome is depicted, the playing of the video is automaticallypaused. On a second screen the user is presented with a set of moralprinciples descriptors (e.g., drop-down lists) to enable the user toselect a first moral principle descriptor 632 and a second moralprinciple descriptor 633. The set of moral principles descriptors may bea subset selected from a wider set of moral principles descriptors. Thesubset is chosen for applicability and ease of distinction (e.g. reducesimilar moral principles). Once the user has selected two moralprinciple descriptors, the second screen indicates which moralprinciples are appropriate (e.g., the second screen displays a firstmoral principle descriptor and a second moral principle descriptordetermined to be most applicable to the moral dilemma). The descriptorsdetermined to be most applicable may be responsive to, for example,expert evaluation and/or community responses. In a preferred embodiment,the playing of the video on the primary screen is automatically resumedat a suitable point prior to the paused position.

Alternatively, or additionally, a user is provided an opportunity toanswer (e.g., YES or NO) the Dilemma question 625. A user's answers aretabulated and summarized at the end of the video to inform the user ofhow the user's decisions compare to the agents in the movie (e.g., whatthe characters decided), and/or compared other users' answer 639 to thegeneralized moral dilemma question. An entertaining psychologicalprofile mat by provided the user that is responsive to the user's andcommunity replies. Additionally, responsive to the user's preferences,Dilemma answers may be posted to social media, and otherwise shared withfriends (e.g., emailed), and the Dilemma community at large. Aggregationof the data may provide a reading on the moral compass of a culture, andother opportunities for research.

In a list mode, Dilemma replay function control 641 enables replayingthe various previously played segments of the video that precede thedecision point and that assist the user in recollecting the relevantdetails and context of a Dilemma. Further, a linkage may be provided toexternal information or discussion on the topic 642.

Trivia, Info, and Identification Second Screen Functions

A video map's Trivia function data comprises a segment definition and/orframe identification, a question, a set of multiple choice answers, anidentification of a correct answer, a write-up, and linkages to relatedinformation. Generally, the point at which a trivia question isdisplayed and the subject matter of the trivia question is related tocontent currently or recently depicted in the playing of the video, suchas a noteworthy moment, item, action, performer, character, location,dialogue, fashion, film production, special effect, and productionerror. Additionally, the subject matter of the trivia question may berelated to topics of general interest or indirectly related to the videosuch as awards, budgets, and performer information.

For example, in the motion picture Captain America: The First Avenger, avideo map identifies a play position in the movie at frame number 69822.The preceding segment, defined by frame numbers 69536-69821, is thefirst depiction in the movie of Steve Rogers in his Captain Americauniform. A trivia question related to the content beingcurrently/recently depicted, comprises the following “Actor Chris Evans,who plays the role of Captain America, also starred as a superhero inwhich Marvel movie?” The answers which in this example, are directlyrelated to the content being depicted comprise: “(A) Fantastic Four asJohnny Storm, (B) Spider-Man as Harry Osborn (C) Daredevil as MattMurdock (D) X-Men Origins as Wade Wilson”.

Additionally or alternatively, the map's Trivia data may be presented asinformation without requiring a user to answer a question. A write-up tosupport the Trivia function may also provide additional information andfacts. For example: “Actor Chris Evans, who plays the role of CaptainAmerica, also starred in the 2005 Marvel movie the Fantastic Four asJohnny Storm, The Human Torch.” The video map also associates the triviaitem with other internal/local or external/remote source of relatedinformation (e.g., Wikipedia website page).

FIG. 7 is an illustration of a second screen display of features of aTrivia function. In this exemplary embodiment, there is no communicationfrom the second screen (e.g., tablet) to the primary screen (e.g.,television). Thus, the depicted interface 701 lacks the remote controlsincluded in other interfaces. The video map is synchronized to the videoplayed on the primary screen, and is thus able to identify the currentplay position in order to perform the various notification functions(e.g., Trivia 711).

In the event that the Trivia function has been activated, and responsiveto the video map identification of when during the video the trivianotification is to be made (e.g., displayed, audible, and/or vibration),at the appropriate point during playback, the Trivia function controlis, for example, highlighted 711. A Trivia Game function comprises adisplay of, for example, a Trivia question 712 and a set of multiplechoices answers 713 (e.g., A B C D). Following a user answer indicatinga selection of one of the answers (e.g., option C), the correct answer714 is identified (e.g., shaded answer). The Trivia functionsupplementary information may further comprise a fuller explanation ofthe correct answer including additional information 715 (e.g., the moviereleased in 2005).

Additionally, a tally of correct answers, asked questions, remainingquestions for the user 716; statistics of a competitor(s) playing on aseparate second screen(s); and statistics relating to how other usershave replied to the question 718 (e.g., percentages in each of theoptions) may be provided. Further, a user's scores across a series ofmovies may be ranked among a circle of friends and the general communityof users. In a competition mode, a time limit to answer a question isimplemented and is response to the expected time required to read thequestion and the optional answers. Further, the trivia questions may beassociated with a level of difficulty and point for a correct answer areawarded 719 based on the level of difficulty (e.g., a 1 being the lowestlevel and a 5 being the highest level) and total points are tabulated.Optionally, a user may specify a level of difficulty as well as thecategory types that are presented. To support the later, a video map'sTrivia data would also categorize (e.g., Cars In Movies) the triviaquestions.

A Trivia function's supplementary information may also comprise a linkto an external/remote source of related information. In that event, anexternal information icon 720 is displayed to indicate the availabilityof additional information relating to the subject of the particulartrivia question (e.g., Chris Evans' Wikipedia page).

Generally, as in other functions, responsive to a user's preestablishedfunction preferences, the display of a trivia information item or atrivia question is responsive to the content being depicted in thevideo. Alternatively, a display of a trivia information or question isresponsive to a user's general preestablished lack of interest in acontent (e.g., Explicit Violence category, commercials) being currentlydepicted. Further, a display of a trivia information or question isdirectly responsive to a user's specific request at a play positionduring a playing of a video. Specifically, while viewing a broadcastmovie comprising commercials (e.g., 30 second promotionaladvertisements), a user's activation of the Trivia function control 711would cause the display of, for example, the next available triviaquestion 712 and corresponding answers 713 in the list of availablequestions, and/or a display of a trivia question 712 and answers 713most relevant to the immediately depicted video content. In anembodiment where the occurrence of a commercial break (interruption)during the playing of a video is identified (e.g., a match or mismatchof the acoustic signature to the acoustic database), and responsive to auser's preestablished preference, a second screen function routine(e.g., trivia questions) is automatically enabled.

Similar to the Trivia function and the Dilemma implementations is apersonality quiz implementation. Preferably in a list embodiment severalpersonality questions are asked to accumulate enough information tocorrelate the user's answers with one of many characters in a video. Avideo map would identify a plurality of characters in a video, andassociate specific answers from a set of multiple choice questions witheach character. The individual answers are drafted, balanced and may beindividually weighted to provide a user, if not an entirely accuratepsychological analysis, at least an entertaining experience. Not everyanswer need be relevant or critical to the analysis. Questions may beassociated with any set of performer/characters (Superheroes in TheAvengers), objects (Cars in The Fast and the Furious), or group(Families in The Godfather). After answering a set number of questionsthe user is informed of which character/object/group their answers areassociated with. Alternatively, the choices a user makes while answeringthe Dilemmas questions can be collected, analyzed and correlated to aperformer/character's traits.

An Info function control 721 provides access to detailed informationabout noteworthy depictions in the movie relating to, for example,items, actions, performer, characters, performer attributes, locations,dialogue, menu items, recipes, world records, and awards. Video map datacreated for, or utilized by, the Info function comprises, for example: asegment definition; a title; a write-up for a noteworthy item depictedwithin a segment of the movie; and a linkage to more detailedinformation. For example, in the motion picture Casino Royal, the videomap defines a segment by frames numbers 106739-106983. The segment,titled “Vesper Martini”, relates to the character James Bond ordering amartini. The write-up to support the Info function comprises thefollowing recipe: “The Vesper Martini: Ingredients: 3 measures ofGordon's Gin, 1 measure of vodka, ½ measure of Kina Lillet, and Lemonpeel for garnish. Directions: Pour the gin, vodka and Kina Lillet into acocktail shaker over ice and shake (don't stir!) well until ice-cold.Strain into a martini glass and add a lemon peel as a garnish. Drink itwith the swagger of a certain British spy, then thwart your enemies andget the girl. All in a day's work.” The video map associates thefollowing information link:http://www.jamesbondlifestyle.com/product/vesper-martini. In one of manypossible embodiments, the display and frequency of Info functioninformation is similar to that of the Plot Info function.

An Identification (ID) function control 722 provides access to a displayof identification information relating to, for example, the depiction ofitems, actions, performer, characters, performer attributes, anddialogue. The ID function control 722 enables selective retrieval ofidentification terms and keywords at a user's requests. Data createdfor, or utilized by, the ID function comprises, for example: a segmentdefinition; a list of keywords for a noteworthy depiction within asegment of the video; the category of the depiction (e.g., automobiles,apparel, martial arts), and a linkage to more detailed information.Examples of ID information include: “Trinity's dark green motorcycleDucati 996” in the movie The Matrix Reloaded; “James Bond's 007 weapongun machinegun SMG HK Heckler and Koch UMP9 with suppressor silencer” inthe movie Casino Royale; Carrie Bradshaw's wedding dress gown coutureVera Wang strapless long frills in white cream in the movie Sex and theCity.

By contrast to the Info function, in a preferred embodiment thefrequency of the availability of ID function information is similar tothat of the Who function. In such as case, there is no need to notifythe user of the availability of information as the video is beingplayed, as the user would activate the ID function when motivated bywhat is being played on the primary screen. Alternatively, responsive toa user's preferences for a particular category of information items(e.g., automobiles, apparel, martial arts) the ID icon 722 on a secondscreen notifies a user that detailed ID information is available for anoteworthy depiction of general interest to the user. Further,responsive to a user's preference, ID information may becontemporaneously displayed on the second screen as the video is beingplayed. If multiple noteworthy items appear at the same time the usermay choose to view any or all of the information.

FIG. 7 also illustrates a number of advantageous second screennotification functions. A Snack notification function 731 informs theuser of an upcoming appropriate moment in the video (e.g., 5 minutesfrom the current play position) for a trip to, for example, therefrigerator to obtain snack. The function may additionally inform theuser as to the length of the particular segment (e.g., 2 minutes) andthe nature of the content (e.g., gratuitous action). Thus, in thisexample, a user wishing to be informed of suitable moments in the video,is informed that in 5 minutes from the current play position, there is a2 minutes window of gratuitous action that can be missed in order to getpopcorn. This particular notification function is particularlyadvantageous in a smartphone application that will silently andinconspicuously inform the user of a suitable restroom break point in alengthy motion picture. The application is useful in a public setting(e.g., theater) as well as in a private gathering of a number ofindividuals. (e.g., friends at a home). The Snack notification functionmay be in active notification mode and/or available in query mode.

As is the case with the Snack notification function, every otherfunction and notification function disclosed herein may alsoadvantageously implement notification routines that provide a runningtime clock to the next item that is responsive to an in-video functionthat is of interest to the user. FIG. 7 illustrates an example in whicha user, having preestablished (e.g. the Settings function control 736)or concurrently established (e.g., pressing the Dilemma function control737) with the playing of a video an interest in a function (e.g.Dilemma), is provided the amount of time to the next information display(e.g. “In 12 Min.” 738) that is responsive to the desired function(e.g., Dilemma 737). A user may activate at any moment a timenotification routine (e.g., pressing a function control icon) to obtainan indication of the amount of time, instances, scenes, and/or chapterstill the display of information for the corresponding function.

A similar routine may also be implemented with other video map segmentdata. For example, while playing the motion picture Matrix Reloaded, auser may desire to know how long before the next martial art scene. Inthat instance, software routines, for example, identify the keywordsassociated with the last completed clip, search the video map data forthe next clip with similar keywords, and convert the segment informationinto a counting down timer. Similarly, every Compilations featurepresentation provides the necessary segment data and presentationidentification to enable a user to establish a preference for a segmentnotification that is responsive to the subject of the compilation.

In a second screen embodiment, a user may desire to only view thesegments in a Compilation titled “Martial Arts” for the motion pictureMatrix Reloaded. In that instance the user activated the Segmentnotification function control 739, which would cause the display of alist of available compilations or searches. Following selection of a setof segments, the second screen function displays the amount of time tothe next segment in the compilation (e.g. “In 15 Min.”), thus enablingthe user to fast forward or skip to the next responsive segment. Whilein the segment, the Segment notification function control 739 mayinstead or additionally display the remaining playing time of thesegment (e.g., as in the Content notification function).

In the example of a Who function, the function control may toggle on andoff responsive to whether information is available or not for acharacter being depicted, or in those instances that a character is notpresent in a clip.

The video map information to drive these and other functions may also bederived from the various presentation features (e.g., 60 MinutesPresentation). Alternatively, video map information is specificallycreated to support a notification function. For each identifieddepiction suitable for a notification, the video map informationprovides an identification of the beginning play position, and durationof the depiction. Optionally, the video map information furthercomprises a few keywords to identify the beginning of the depiction, awrite-up of the depicted content, a rating of the significance if any tothe plot, and/or a content category.

Advantageously, the rating and category data enable identifying moresuitable points in the video, and enable the implementation of a Snacknotification function that is responsive to a user's preestablishedpreferences. For example, a user may preestablish which time period(s)of the video the notification function is active (e.g., 15 minutes fromthe start to 30 minutes from the end excluding credits), the frequencyof notification (e.g. every 10 or 30 minutes), the type of notification(e.g., audio tone, vibration, lock screen alert, icon highlighted, icontimer 731), the minimum period of time that is required (e.g., 5minutes), and. advantageously, which categories of content are lessimportant to the user (e.g., Mushiness). The rating, which maybe assimple as an snack icon or bathroom icon, is particularly useful when,for example, the motivation for querying the feature is a snack breakrather than a bathroom break.

A Content notification function 732 informs the user that an upcomingpotentially objectionable content will be displayed on the primaryscreen (e.g., 12 seconds from the current play position). The functionmay provide the user additional information 733, such as the length ofthe particular segment (e.g., 13 seconds), the category of the content(e.g., graphic violence), and responsive to the user's preference plotrelevant information. Upon notification the user may elect to focusattention of the second screen display during the period of time thatthe Content notification control is highlighted. Advantageously, theContent notification control 732 provides a time remaining countdownand/or written description relating to any information that may besignificant to the plot.

As in other second screen functions, a user is provided a range ofoptions for preestablishing preferences for this function. Specifically,with respect to the Content notification function, a content preferenceinterface similar to that previously depicted may be utilized to enablea user to preestablish which categories and levels of explicitness theviewer would prefer to receive Content notifications. The video mapinformation to drive this function may be derived from the Controlpresentation feature. Alternatively, segment information identifying abeginning and an ending position in each of a plurality of segments, andeach associated with a content category and level of explicitness, arespecifically created to support the Content notification function.Advantageously, the latter implementation provides a more preciseidentification of the actual video frames of potentially objectionablecontent since there is no requirement to consider the quality of theedit if the content were excluded by the Control playback feature. Stillfurther, segment information may be associated with keywords identifyingthe content to enable a user to preestablish notification parameters bythe use of keywords. For example, a user, that deems the depictions ofsnakes to be objectionable, would enter appropriate keywords to inhibitthe display of those types of depictions.

The interface illustrated in FIG. 7 also provides the means for a userto facilitate and/or perform the synchronization of the processing ofthe video map to the playing of the video on the primary screen. Inthose instances that a video timecode may not be displayable to the useron the primary screen, a user may utilize the continuous display of IDinformation and/or video frame thumbnails included with and provided bythe video map to synchronize the processing of the video map. Ascrollbar 741, thumb 742, and discreet adjustment buttons 743 enablesthe user to adjust the second screen internal clock current playposition 744 to match or effectively approximate the actual playposition of the video on the primary screen. Additionally oralternatively, in those instances that a video timecode (HH:MM:SS) isdisplayable to the user on the primary screen, a user may directlyenter/adjust 744 the timecode on the second screen.

Importantly, the illustration of FIG. 7 serves to illuminate the manypotential synergies that are available by the combination of functionsin a second screen embodiment. For example, a user may preestablish apreference for Control function notification 732, and, when the time ofthe potentially objectionable content permits, to be automaticallypresented with, for example, a trivia question 712, or other informationavailable from any other function or external resource. Further, a user,while playing the movie, the user may save an item of interest to beautomatically displayed when content of no interest to the user is beingdisplayed on the primary screen. For example, a user may save Shoppingfunction items and Locations function items, precisely for that purpose.In essence, the user dynamically creates commercial interruptions, at auser responsive suitable points in the video, for a product of interestto the user,

Shopping/Music Second Screen Function

The Shopping function provides information and access to resources,e.g., websites, relating to a currently and/or recently displayednoteworthy item, prop, or location. Website linkages provide additionalinformation and/or the opportunity to purchase a depicted item or obtaina service associated with the depicted location. Examples of noteworthyShopping items and services include: a life size statue of the T-800statue from Terminator, the Algerian Love Knot necklace from CasinoRoyale, Arthur's leather jacket from Inception, and a James Bond bungeejump experience from Golden Eye. The Music function identifies the titleof the musical item, the performer, the album, other musical itemidentifying information, and includes links to a website that canprovide additional information and/or the opportunity to purchase themusical item.

With respect to a Shopping/Music function, the disclosures in¶å0056-0076, and ¶å0083-0116 of U.S. patent publication 20130251337,titled “Providing Item Information During Video Playing” areincorporated herein by reference. With respect to a Shopping/Musicfunction in a second screen embodiment, previously incorporated U.S.patent application 20120151509 is incorporated herein by reference.

FIG. 8A is an illustration of a second screen display of features of aShopping function. In this particular embodiment, contemporaneouslywith, or alternatively to, the display on a primary screen of anindication that information is available, an identification of each ofthe items on which information is available is displayed on thesecondary screen 801. In this particular example, the video at thisparticular play position is depicting three characters: a boy holdingballoons, a girl wearing a dress, and a man wearing a hat. In thisinstance, the play location is monitored for the presence of apurchasable item. The current play location within a video is monitoredand a plurality of segment definitions are searched to identify asegment definition that is responsive to the play location.

When a segment definition or frame identifier is responsive to the playlocation, and during the period defined by the segment definition orassociated with the frame identifier, an indication is displayed thatitem information is available for an item or items being depicted. Inthis example, an indication that Shopping information and Locationsinformation are provided on the second screen. Although it may not benecessary, in this embodiment, as in previous examples, the display ofShopping function control 811 and the Locations function control 812 arehighlighted. FIG. 8A illustrates an identification of multiple Shoppingitems (i.e., the 90 Day Balloons 832, the Strip Dress 842, and the TopHat 843) and one Locations item (i.e., The Gotham National Bank 844).

As in previous examples, the identification of the item may beassociated with additional information about the item (e.g., acomplementary item, a service related to the product, and insurance).Item identifying information comprises, for example, the informationcategory 831 (e.g., Shopping icon), an image or video of the itemextracted from within the video 832, and textual identification of theitem “90 Day Balloons” 833.

Item identification information (832, 843, 843, and 844) may also serveas a means (e.g., clickable buttons) to enable the user to requestadditional item information relating to a displayed item information. Anitem information may constitute a button/object associated with a“clickable” event that when activated would launch the additionalinformation routines. In response to receiving such a request foradditional information for an identified item, additional iteminformation, such as purchase information is provided. Many sourceseither provided or integrated with the video or external to the videomay provide supplementary information. Additional item information maybe downloaded from a suitable website by means of a link or referencedata provided with information associated with, for example, the segmentdefinition or item identifier. Retrieval of additional information maybe real-time, or non-real-time and pre-stored in volatile ornon-volatile memory for use when needed. While supplementary informationcomprise many items and categories of additional item information, thereis no requirement that supplementary information itself provide anyspecific information.

The interface, taking advantage of the Search feature of the secondscreen application, also provides the user the capability toautomatically search 851 for instances of the same currently highlighteditem (e.g., 90 Day Balloons). Further, in this exemplary embodiment, auser is also provided the capabilities to specify and/or limit/filterthe display of information responsive to the user's shopping category,sub-category, or item preferences 855 (e.g., Arts & Crafts, PartySupplies, Party Balloons). Similarly, a user is also provided the means(e.g., Internet icon control 852) to obtain external information ordiscussion on the topic (e.g., website page), and the means (e.g., SUBicon control 853) to play a presentation of the plurality of videosegments (i.e., a Subject) within the video relating to the currentlyhighlighted item.

Advantageously, the item identification routines are configured tosearch a plurality of segment definitions to identify segmentdefinitions that are responsive to the request location and apredetermined play period prior to the request location. To accommodatethose situations, in which at the time that the user has requested iteminformation, the request location may be adjusted to include the requestlocation and a predetermined play period prior to the request location,for example, 30 seconds. In this fashion, a segment definition may beresponsive to the information request or a predetermined play periodprior to the request location. Advantageously, this method enables theretrieval and display of item information associated with a number ofrecently played depictions of items having item information available.

A second screen embodiment elegantly supports a continuous display ofidentification information contemporaneously with the playing of thevideo. The display of identification information need not require thepausing of the playing of the video. Alternatively, as a user predefinedoption, the playing of the video may be automatically paused andreplaced with the display of the additional item information, while thesecond screen continues to provide access to the various itemidentification information controls (832, 843, 843, and 844). In asecond alternative, when a user requests additional information, theplaying of the video may be automatically paused, and the display on thesecond screen may be replaced with the display of the additional iteminformation. Any number of permutations between what is displayed on theprimary and secondary screen, and in which sequence, are possible.

In an exemplary embodiment, item identification information isassociated with a unique identifier. When a user requests additionalinformation, the user's apparatus capable of processing data andinstructions executable by a processor, performs the step of providingthe unique item identifier to a source remote to the apparatus that isdedicated to maintaining updated detailed additional item informationspecifically produced and edited for display within the video. Theunique item identifier may be accompanied with or incorporate videoidentification and play location identifiers to obtain additional iteminformation that is responsive to the play location within theparticular video. For example, a necklace may appear in connection withdifferent garments, e.g., the Algerian Love Knot in the motion pictureCasino Royale is worn by the character with different garments and indifferent locations. The additional information that may be providedwith the necklace item information would be responsive to the particulargarment, situation, and/or location depicted in the video at the momentthat the user request was received for item information in response tothe display of an indication that item information is available. When auser elects to exit the additional information environment (e.g., a pagein a website or the Exit control key) a number of different options maybe executed as to how the playing of the video is resumed. The ¶0168disclosures and video playback resuming disclosures in the'721publication are incorporated herein by reference.

A user may preestablish a preference for the types of shoppingcategories for which a notification is to be provided and/or for whichthe retrieval of additional information is to be performedautomatically. For example, a user may prefer Shopping notification foronly items of jewelry and the automatic retrieval of a website page forthe designer version of a jewelry item (e.g., higher priced) rather thanthe costume jewelry interpretation of the item (e.g., lower priced). TheAlgerian Love Knot necklace depicted in the motion picture Casino Royaleis available from the designer Sophie Harley, and an imitation isavailable from a costume jewelry website at a fraction of the price.

FIG. 8B is an illustration of a second screen display of features of theShopping function simultaneous display of multi-sourced externalinformation. FIG. 8B illustrates an iPad second screen 870 embodimentthat is responsive to a user enabling automatic retrieval of theShopping external information for items as they are displayed on theprimary screen, or that is responsive to a user activating the retrievalof the Shopping external information following notification of Shoppinginformation for an item displayed on the primary screen. Responsive tothe depiction of the Algerian Love Knot necklace in the motion pictureCasino Royale, the Shopping function routines retrieve and display aresponsive page from sophieharley.com website 871 depicting the designerjewelry item, from the designer Sophie Harley, at a price at £2,280.00.The Shopping function routines have also contemporaneously retrieved anddisplayed a responsive page from emitations.com website 875 depictingthe costume jewelry version of the item at a price at $176.00.

The Shopping function routines, which, for example, automaticallyretrieve and display multiple external information, are not limited asto which specific items are displayed. For example, website page of anitem (e.g., a blouse) from one merchant may be simultaneously displayedin combination with multiple website pages of complementary items (e.g.,a skirt, a silk scarf, an item of jewelry) from the same or differentmerchants. Similarly, the Shopping function routines are not limited asto the type/format of the information. Combinations of videos, images,website pages, multiple pages from the same website, and/or applicationsmay be utilized. For example, a website page of an item may be combinedwith an instructional video of the item from youtube.com.

Exemplary embodiments comprise: receiving, on a second screen device,synchronizing information that is responsive to a playing of a video ona primary screen device; establishing, responsive to the synchronizinginformation, a synchronization to the playing of the video; retrieving afirst linkage corresponding to a purchasable item depicted within thevideo, and a second linkage corresponding to a second purchasable itemrelated to the depicted purchasable item; and downloading, responsive tothe first linkage and to the second linkage, and simultaneouslydisplaying on the second screen device, information (e.g., a websitepage) corresponding to the purchasable item depicted within the video(e.g., the original, authentic, designer, real, fine, or higher pricedversion of the depicted purchasable item), and information correspondingto the second purchasable item (e.g., copy, knock-off, costume, lowerpriced, alternative, or complementary product), the displaying on thesecond screen device being responsive to a time period offsetsynchronization to the video playing on the primary screen device. In anembodiment where the information is a website page, each correspondingpage is independently navigable, and, optionally, each provides a userwith the opportunity to place an order for the item directly with themerchant. A designer item (e.g., clothing) usually bears the logo of, oris associated with, a recognizable fashion designer. A costume jewelry,for example, is usually an ornament, accessory, article, or detail usedto beautify the appearance of something to which it is added.

As a user views the movie, the user may save items of interest tocollections (e.g., “bags”) that they name and categorize. At a moreconvenient time, the user can then review the collected of items, obtainadditional information, place orders, post to social media, andotherwise share with friends.

Still further, a user may elect to be notified that an item of interestis available at a local brick and mortar store, and/or, for example, onsale at a preestablished price or discount threshold (e.g., 30% MSRP).Taking advantage of the GPS functions of second screen device, a usermay elect to be notified when in proximity to a brick and mortar storethat has the item available. To support this function, the video map mayfurther associate with the item information and linkages, the name ofthe retailer at which the item and other similar items may be inspectedand purchased. The notification may provide a clip from the videofeaturing the item, an image of the item, other previously retrievedinformation, together with any notes or reminders the user may havecreated with respect to the particular item.

Control and Preview Second Screen Functions

The Control function enables the customization of a presentationaccording to a user's content preferences for the level of explicitnessin each of a plurality of categories of possibly objectionable content.The application of a user's content preferences to the video map of avideo results in the automated logical selection of sequential andnon-sequential segments of the selected video that are consistent withthe user's content preferences. Any segment, with a content codinghigher (abstract) than the user-selected content preference for thecorresponding category would not be included in the video produced forthe user. The segment selected for viewing would satisfy thepresentation preference and would have a coding level equal to or lowerthan the user specified content preference for that category. If, forexample, a user selects, the 60 Minutes presentation and also requestthat the user's content preferences for the level of explicitness ineach of fourteen categories be applied to that presentation, then theresulting “Custom 60 Minutes” presentation may be reduced to 55 minutes.

A viewer may preestablish content preferences prior to playing anyvideo, prior to playing a specific video, and may modify the contentpreferences during a playing of the video with the new contentpreferences being applied as the playing of the video resumes orcontinues. Advantageously, video specific content preferences willdisplay only the options for which content is actually present in thevideo. For example, if the video does not contain any depictions ofGore, the Acceptable Level Of Explicitness will be set at Noneirrespective of any generalized preestablished content preference.

Advantageously, a user may preestablish different set of contentpreferences to be applied to the different features and functions. Forexample, a user may prefer that the Presentations (e.g., Action and 60Minutes) be responsive to the content preferences, but would prefer thatno content preferences or limited content preferences be applied whenthe Search function or Best Lines feature are being utilized. Stillfurther, content preferences may be preestablished broadly,preestablished separately for various genres, types, and classes ofvideos, and contemporaneously and specifically to a video. A user maypreestablish different set of content preferences to be applied to thedifferent genres of motion pictures. For example, the set of contentpreferences applied to a romance would be different than the set appliedto an action movie. Still further, individualized set of contentpreferences may be established for each of a plurality of users in ahousehold, be password protected, with supervisor limits set bycategory.

A Control second screen function advantageously provides for anadjustment of content preferences without interruption of the playing ofthe video. FIG. 9A is an illustration of a second screen interface forestablishing content preferences. The content preference interface 900enables a user to establish content preferences on a second screenduring a playing of the video on a primary screen. A second screen userinterface embodiment advantageously provides the means for touch screencontrols as are available in smartphones and tablets.

A second screen embodiment, as illustrated in FIG. 9A, combines acontent preference selection interface and remote control capabilities.This particular second screen embodiment, as an example of the variouspossible combination of elements, relocates the Who in-video functioncontrol 901 to the right of the playback function controls 902, andincludes a Control function control 903, a preview function control 904,video timeline navigation control 905, and a current play positionidentifier 906 (e.g., current play time in an HH:MM:SS:FF format) and/orremaining playtime from the current position.

The Control function control 903 enables the display of a Controlinterface. The Control interface enables a user to establish contentpreferences for the level of explicitness 911 (i.e., None, Mild,Explicit, or Graphic) in each of fourteen categories of possibleobjectionable content 912 (i.e., Bigotry, Blasphemy, Bodily Functions,Dishonor Flag, Dishonor Parent, Gore, Hideous, Mature Subjects, Nudity,Sex, Substance Abuse, Violence, Violence to Animals, and Vulgarity). Inthe example illustrated in FIG. 9A, a user indicates content preferencesby touching a particular button 913 associated with a preferred level ofexplicitness. The user in this case has elected to omit depictions ofDishonor Flag during a playing of the video.

In this embodiment, the Control second screen interface 900 combines aControl content preferences matrix 911-912 with a description 921 of thevarious levels for a currently selected, last selected, or otherwiseindicated content category 913. A context responsive help control orsecond touching of the Control function control 903, would provide thedescription 921 summarizing the nature of the potentially objectionablecontent that is excluded at a particular level of explicitness. An Exitfunction 909 activates the display of a generalized interface.

A novel and material advantage of a second screen embodiment forestablishing content preferences is that a user may adjust contentpreferences without interrupting the viewing by others and withoutinforming others of the changes. This is especially advantageous whilechildren may be viewing a movie and a parent or supervisor determinesthat the incidence of certain content (e.g., Explicit Violence) isdeemed excessive.

FIG. 9B is an illustration of a second screen interface for establishingcontent preferences and previewing content. This interface 950 combinesa content preference selection matrix with information indicating theamount of possibly objectionable content in a level of explicitness fora content category. This particular matrix advantageously displays,responsive to the content preferences of a user, how much possiblyobjectionable content remains to be played from the current position951. The information may be provided in terms of instances and/orduration, e.g., accumulated time in minutes and seconds 952. To furtherexplain, at twelve minutes, seven seconds, and two frames into the video953, for a user with a preference for an Explicit level of Violence, sixminutes and fifty seconds of Explicit Violence and seven minutes and sixseconds of Mild Violence will be depicted, and Graphic Violence will notbe depicted. In this particular embodiment, a user may increase ordecrease a level of explicitness by toggling a level of explicitness ina content category.

While, the principal object of the video map data relating topotentially objectionable content is to support a user's preferences toinhibit the playing of content generally deemed objectionable by theuser, another object of the data is to enable a user to preview and playthe potentially objectionable content. Activating the Preview functioncontrol 962 enables the Preview function interface features androutines. Activating the Control function control 961 returns theinterface to content preferences establishing mode. The playbackroutines associated with, for example, the content preference selectioninterface, provides an elegant means to cause the playback of thecontent associated with any desired category or level of explicitness.For example, a viewer wishing to preview the forthcoming potentiallyobjectionable content categorized as Explicit Violence, would touch thecorresponding level of explicitness control 952 and then touch a forwardsearch Preview function control 971. A video search Preview functioncontrol 972 activates routines that retrieve all the responsivedepictions within the video. A backward search Preview function control973 provides a backward limited retrieval.

In this example, responsive to user preferences and the particularsecond screen embodiment, following activation of the forward searchPreview function control 971, the playing of the video on the primaryscreen is automatically paused, and the content categorized as ExplicitViolence that takes place from the current play position 953 (e.g.00:12:07:02) is played in the primary screen and/or in a window 979 onthe second screen. In other words, advantageously a look forwardanticipatory preview is limited to segments including depictions with atime stamp in the future of the current play position 953.

The advantages of a look forward anticipatory preview search are furtherenhanced by another embodiment, in which the playing of the video on theprimary screen is not paused, and advantageously a list and description(e.g., text, image, and duration), of each of the segments depicting thepotentially objectionable Explicit Violence, is displayed on the secondscreen. This will permit a user (e.g., parent or supervisor) to adjustcontent preferences to be applied to the balance of the playing of thevideo, without disturbing the viewing by other users, and withoutnecessarily being subjected to the actual depictions of the possiblyobjectionable content.

Ratings Second Screen Function

FIG. 9C is an illustration of a second screen display of features of aRatings function. A video map identifies noteworthy best lines, bestperformances, memorable moments, and romantic moments. The video map'ssegment definitions, descriptors (e.g., Best Lines), and descriptions(e.g., Al Pacino: “My father made him an offer he couldn't refuse.”) areutilized to identify them to a user as they are played and provide theuser of the opportunity to rate them (e.g., 3 stars). In one embodiment,a video map associates an image with the identification of thenoteworthy lines, performances, and moments.

Generally, the image (e.g. snapshot selected from within the movie)corresponds to the item identified (e.g., a best line is associated withan image of the character delivering the line).

However, in an alternate or complementary embodiment, a display of bestline is enhanced (e.g., made more entertaining) by a juxtaposition withan image used to create synergy between the best line and the image. Incertain instances a best line is advantageously associated with aniconic image (e.g., a single frame of video selected from anyplacewithin the video) related to the subject of, and/or characterdelivering, the best line. A classic example of this is in movie TheGodfather, the memorable line “It's a Sicilian message. It means LucaBrasi sleeps with the fishes,” is associated with and displayed over animage of the fish wrapped in body armor. An iconic best line whichdefines a character may also be associated a displayed over an iconicimage of the character that delivered the line (e.g., the line “I'll beback,” is displayed with the image of the Terminator looming over thepolice kiosk). Still further, a humorous juxtaposition can often becreated by combining a dramatic best line with an unexpected image. Forexample, a best line “meme” is created by overlaying a best line over animage. In those instances that a selective retrieval and playback ofvideo is available, a line, performance, or moment may be associatedwith a playing of a video segment and the display may loop the videocreating an animated gif. In advanced embodiments, a user is providedthe routines to facilitate the selection of images and/or video,juxtapose them with an item (e.g., overlay a best line), and distributeto others (e.g. publish in social media platforms).

An exemplary second screen interface 981 includes Ratings functioncontrol 982. As in other functions, a user may activate the function ata current play position by touching the Ratings function control 982.Alternatively, a user may activate the function for the entire video bytouching and momentarily holding the Ratings function control 982. Whenactivated, the current play position is compared to the video map toretrieve and display responsive information corresponding to Best Lines983, Best Performances 984, Memorable Moments 985, and Romantic Moments986 being currently played. The displayed information comprises adescription 988, which, in an example where a scene/clip includes a bestline, the description identifies the performer of the best line and atranscript of the line. The illustrative example also indicates that thesame clip may provide items in each of multiple categories. In the caseof, for example, the Best Lines category, a clip may be associated withmultiple entries. A multiple performers usually share, for example, aRomantic Moment.

An identified item is associated with a rating interactively assigned bythe user 987. The rating may be shared with the user base at large or acircle of friends to compile a community/friends rating which may thenbe additionally displayed. The Ratings feature is particularly wellsuited for sharing and posting information (e.g., movie title, category,play time, duration, description, rating, image, and/or video clip). Tothat extent, and responsive to user preferences, a number ofcommunication functions may be provided. For example, dragging aparticular item to the messaging function control (e.g., email functionicon 993) would enable the transmission of the information by aparticular preselected communication function. Similarly, Ratingsfunction information may be posted in various social media platforms(e.g., Facebook Like icon 994). An alternate and simplified ratingsystem may also enabled 989 and/or user community feedback. Further, auser may save any information (e.g., the Movie title, text of a BestLine, and play position) by touching a category 991 anddragging/dropping to a folder 992 for later viewing or reference.

Search Second Screen Functions

A video map's keywords and search data enables a Search function thatprovides keyword search capability relating to, for example, thedepiction of items, actions, performer, characters, and performerattributes, film technique, production errors, locations, best lines,and dialogue. The Search function enables the selective retrieval ofsegments, clips, and/or scenes responsive to a users search terms andsearch requests. A Search segment is 5-10 seconds, a clip is 30-60seconds, and a scene is 2-5 minutes. Examples of keywords and searchterms include: “Trinity Ducati 996” in The Matrix Reloaded; “Spider-Mankiss MJ” in Spider-Man; “cinematic wipes” in Star Wars; “Brooklyn Deckercleavage” in Battleship; and “bloody horse head” in The Godfather.

In addition to keywords and segment, clip, and scene definitions, datacreated for, or utilized by, the Search function also comprises, forexample, key search terms, a title, an identification of arepresentative video frame, write-up to provide an in-video briefinformation about the item, a linkage to more detailed information, andidentification of related map content (e.g., a Subject). For example, inthe motion picture Casino Royal, the video map defines a segment byframes numbers 106739-106983. The segment, titled “Vesper Martini”,relates to the character James Bond ordering a martini. The write-up tosupport the Search function comprises the following recipe: “The VesperMartini: Ingredients: 3 measures of Gordon's Gin, 1 measure of vodka, ½measure of Kina Lillet, and Lemon peel for garnish. Directions: Pour thegin, vodka and Kina Lillet into a cocktail shaker over ice and shake(don't stir!) well until ice-cold. Strain into a martini glass and add alemon peel as a garnish. Drink it with the swagger of a certain Britishspy, then thwart your enemies and get the girl. All in a day's work.”The video map associates the following information link:http://www.jamesbondlifestyle.com/product/vesper-martini.

A search methodology may be conceptually similar to that implemented by,for example, a Google search engine. Additionally or alternatively, asearch may be facilitated by the presentation of various keywordlistings that are responsive to the depicted items, actions, performer,characters, and performer attributes, film technique, production errors,locations, best lines, and dialogue. Further, a search is furtherenhanced by video map data structures that establish logicalrelationships among segments. The Video Map ¶å0175-0196 disclosures inthe '721 publication are incorporated herein by reference.

Since a search may be performed at any time, there is no obvious need todisplay on a primary screen an indication that a search or searchinformation is available. However, advantageously, a video map mayfurther identify particularly noteworthy searches to be performed. Forexample, the search terms “Bond order drink” retrieves three differentnoteworthy segments from within the motion picture Casino Royal. Thus,an indication that a search is available and recommended for aparticularly noteworthy item is advantageously displayed on a primaryscreen and/or a secondary screen. Responsive to a user's preferences,the recommended searches automatically cause a tiles/list display of theitems on a second screen.

FIG. 10A is an illustration of a second screen display of features of aSearch function. In this particular embodiment, the user has opted todisplay only the function controls for which information is available atthe current play location. In this example, the Control 1011, Music1012, and Search 1013 functions are available. Responsive topreestablished user preference and the current play position, Searchdata is displayed on the second screen contemporaneously with theplaying of the video on the primary screen. The displayed Search datacomprises, for example, the title of the depiction 1021, a write up1022, and key search terms 1023.

The Search feature interface also provides the user the capability toautomatically search for depictions similarly keyworded. In thisexample, key search terms 1023 are automatically entered in the searchbar for the user. The user may perform a search by using the Searchcontrol 1024. The search results may be displayed on the primary screenand/or in the second screen.

As is customary with second screen devices, a Search function interfacealso provides a QWERTY keyboard layout as generally implemented insmartphones and tablets. The keyboard facilitates the entry of searchterms. In the example of FIG. 10A, key search terms 1023 areautomatically entered in the search bar for the user, and may bemodified by the use of the keyboard. The user may cause a search byusing the Search control 1024 or alternatively a backward limited searchcontrol 1025. Responsive to the results of the search, identification ofthe matching segments are provided. In this example, the search resultedin the identification (e.g., title, representative image, andHH:MM:SS:FF location in the video) of multiple items (e.g., Rum And Soda1041, Vesper Martini 1042, and Shake And Not . . . 1043).

A user may desire, during a playing of a video, to search a specificdepiction relevant to the current play position that occurred prior tothe current play position. This would enable replaying portion of thevideo of interest without being subjected to spoiler depictions.Advantageously, the Search second screen interface 1001 may also providea backward limited search 1025 to restrict search results to the portionof the video that preceded the current play position. A backward limitedsearch may also be referred to as a forward excluded search. If thebackward limited search control 1025 is activated, then only the keyworddata associated with segments that precede the current play position aresearched to identify the segments/scene/clips that are responsive to thesearch terms. Following the playing of the depictions that result fromthe search function, the playing of the video is resumed at, forexample, a location that is responsive to the location at which theSearch function was initiated.

The search interface also includes a forward limited search 1026. If theforward limited search control 1026 is activated, then only the keyworddata associated with segments that follow the current play position aresearched to identify the segments/scene/clips that are responsive to thesearch terms. Similarly, a user is provided the means (e.g., Interneticon control 1031) to obtain external information related to the search(e.g., website page), and the means (e.g., SUB icon control 1032) toplay a presentation of the plurality of video segments (i.e., a Subject)within the video relating to the currently displayed segment.

FIG. 10B is an illustration of second screen display of features of acontent-categorized Search function which comprises routines thatfacilitate the retrieval and display of segments depicting contentrelating to specific performers/characters and/or contentclassifications. The Search function interface may as simple as anadapted version of the Control function matrix 1051 that enablesselecting a category (e.g., Sex) and a level of explicitness. Forexample, a viewer, wishing to play the content categorized as ExplicitSex, would touch the corresponding level of explicitness control 1052and then touch a video Search function control 1055. Such a simplifiedspecialized Search function facilitates the retrieval and display ofsegments depicting, for example, sexy, sensual, erotic, and/orpornographic content.

As is the case with the Preview function search controls, the Searchcontrols additionally include a backward limited Search function control1056 that activates routines that restrict search results to the portionof the video that preceded the current play position 1053. If thebackward limited search control 1056 is activated, then only the dataassociated with segments that precede the current play position aresearched to identify the segments/scene/clips that are responsive to thespecified search. A backward limited search during playback onlyretrieves and plays video clips from the selected point backwards so asto not spoil upcoming events. Following the playing of the depictionsthat result from the search function, the playing of the video willusually be resumed at a location that is responsive to the location atwhich the search function was initiated. Similarly, a forward limitedSearch function control 1057 activates routines that restrict searchresults to the portion of the video that follows the current playposition 1053.

Advantageously, a specialized Search function interface utilizes a setof classification controlled dropdown menus and/or classificationcontrolled auto-complete fields. In this particular example, the searchSex classification 1060 is associated with multiple drop-down listsand/or auto-complete fields including: a first performer 1061 and thedesired physical attributes of the first performer 1062, a secondperformer 1063 and the desired physical attributes of the secondperformer 1064, an act 1065 (e.g. kissing), and the position 1066 (e.g.,standing). Clearly not all fields need be completed, nor are theylimited to the generalized labels. For example, the field labeledPerformer 1 includes a gender entry option rather than requiring entryof a specific performer/character. Further, typical functionsimplemented in complex searches may be implemented (e.g., advancedBoolean searches). These advanced searches facilitate an unparallellevel of discreet searches and retrieval of content for a specifiedcombination of performer(s)/character(s)/gender 1061 1063, physicalattributes 1062 1064, acts 1065, and/or positions 1066.

Advantageously, a user is also provided an external search function(e.g., search other motion pictures 1068 stored locally or remotely, andan Internet icon control 1069) to conduct an automated external searchof the search terms entered in the various controlled drop-down listsand/or auto-complete fields. The search results may be displayed on theprimary screen and/or in the second screen.

Data created for, or utilized by, the specialized Search functioncomprises, for example: segment definitions; identification of theperformers/characters; and lists of keywords for a noteworthyattributes, clothing, actions, and positions within a segment of thevideo. Examples of Preview function data include: in the movie 300 Riseof an Empire, a segment defines by frames 80165-80290 and descriptors“Eva Green as Artemisia naked topless large natural breasts”; and in themovie Captain America: The Winter Soldier at a segment defines by frames2013-2104 and descriptors “Chris Evans as Steve Rogers large chestmuscles pectorals pecs tight Under Armour V neck fitted shirt”.

Information Display Modes

Generally, it is advantageous that a user be provided the option todetermine the orientation of the second screen, and the inclusion andlayout of the individual feature elements, controls, and/or sets ofcontrols as the user may deem advantageous from time to time. Theillustrations provide some of a nearly infinite number of possibleconfigurations, inclusion of elements, notifications, functions, and thedisplay of information. Advantageously, a user is provided a range ofinformation display modes for which the user may preestablish globaland/or function specific preferences. Still further, taking advantage ofthe optical capabilities of a second screen, the brightness andbackgrounds colors of the second screen interface may be automaticallyadjusted to complement the general/ambient lighting, and, when thelighting permits, the dominant colors of the video being played on theprimary screen.

An information display mode may be as simple as the display of textand/or an icon that awakens a second screen from sleep mode whilemaintaining a no noise and low light condition. In that mode, while thevideo is playing on the primary screen, the second screen, while thesecond screen information is executing in the background, the display isin a sleep state. When a notification event is encountered, a notifyingmessage and/or icon are displayed. Sliding the text or touching the iconlaunches function routines that provide responsive information.

Others exemplary embodiments, illustrate more complex configurationscomprising multiple notification icons, each with behaviors andinformation responsive to the respective information functions andsynchronization parameters. An information active function control maybe highlighted. Others may display functional information (e.g., Dilemmanotification countdown clock), and others may toggle on and offresponsive to the availability of information (e.g., Who functioncontrol).

An information display mode may also take the form of a display ofmultiple icons and information tiles. The display of tiles in a stripand other configurations is well developed in second screenapplications. FIG. 11A is an illustration of a second screen display offeatures of a function in a tiles mode. In this interface example 1101,a Lists/Tiles function control 1102 provides access the various featuresin a list or tiles modes. The Lists/Tile functions may be accessed priorto, during, and/or following the playing of the video. Once activated,toggling the Lists/Tiles function control 1102 switches the display fromthe depicted tiles mode 1103 to a list mode. In this instance, thedisplayed icon and descriptor 1102 is the reverse of the activefunction. In this particular illustration, a user has activated, in atiles mode, the Sexy function control 1104 which facilitates theretrieval and display of images/video depicting sexy, sensual, erotic,and/or pornographic content.

In this exemplary embodiment, a user is also provided the capabilitiesto specify and/or limit/filter the display of information (e.g., Image1105, Performer/Character Name 1106, and video play position identifiersuch as time code 1107) responsive to the user's preferences 1108 (e.g.,Performer, Character, Attributes, and approximate or range playpositions HH:MM:SS). Additionally. or alternatively, thePerformer/Character Name 1106, and video play position identifier suchas time code 1107 are displayed in the selection boxes 1108 as aparticular title is highlighted 1109.

FIG. 11B is an illustration of a second screen display of features of afunction in a list mode. A List second screen interface 1111 enables auser to access the information available for the various second screenfunctions in a list display mode. In this particular example, a user haselected, by pressing the Shop function control 1113, to cause thedisplay of a list of the items 1121 associated with the Shopping secondscreen function. The currently highlighted Shopping item 1122 isassociated with an image 1123 of the item, with additional internalinformation 1131 (e.g., information provided by the video map such asthe time code of the segment depicting the highlighted item), and withadditional external information 1132 (e.g., website page). A user maytoggle to a tile mode by activating the Tiles/List function control1112.

By contrast to the simple information display mode comprising thedisplay of text and/or an icon that awakens a second screen, A farricher and more active information mode would cause a continuous displayof detail information responsive to the various user functionpreferences. For example, the set of geographical maps may beautomatically retrieved and displayed on the second screen as the videocontinues to be played on the primary screen. As was previouslydetailed, this information display mode, advantageously includes contextsensitive controls that also serve to identify the availability ofinformation that is responsive to the play location. In this example,responsive to what is depicted in the movie the context sensitivecontrols include, for example, a Shop control, a Locations control, anda Filmmaking control.

Multi-User Embodiments

The communication capabilities of second screen devices may be utilizedto further realize the potential of the second screen in-videofunctions. A second screen display of integrated video playback andin-video functions may be transmitted and played on the primary screenby the use of a number of devices. For example, Google's Chromecast is athumb-sized media streaming device that plugs into the HDMI port on theprimary screen (e.g., television). A user may utilize, for example, asmartphone, tablet, laptop, to transmit entertainment and applicationsto the primary screen. (See, for example,http://en.wikipedia.org/wiki/Chromecast) Advantageously, mirroring thesecond screen display provides others the available options whileretaining the advantages of using the second screen to navigate. TheApple AirPlay Mirroring implementation enables a user to mirror contentfrom a user's iPhone, iPad, or iPod touch to the user's HDTV via anApple TV device. (See, for example,http://en.wikipedia.org/wiki/AirPlay) The teachings of Google'sChromecast and Apple's AirDrop and AirPlay are incorporated herein byreference. Thus, with the capability of casting/mirroring the secondscreen content, a user may share second screen content with othersviewing the primary screen and/or using other second screen devices.

Further, the communication capabilities of second screen devices andsecond screen functions may be advantageously utilized in a multi-usermulti-second screens embodiments. For, example, a first user and asecond user remote from each other are concurrently viewing the samemotion picture on their primary screen and each has enabled theindividual in-video function of general interest. A first user mayactivate the Info function on his/her second screen, while the remotesecond user may activate the Locations and Music functions on his/hersecond screen. In this example, the first user following reading a Infowrite-up may deem desirable to share that information with the seconduser. With that object, the two second screen devices may havepreestablished or establish a continuous, nearly continuous,intermittent, or temporary communication linkage to share second screeninformation.

Many technologies may be implemented to establish such communications,such as local area wireless technology (e.g., Wi-Fi) as in the case ofclose proximity, and communications networks (e.g., the internet).Further, the transmission of information between the second screendevices may be facilitated by a services provider that provides mapprocessing and/or the in-video information to the users. For example, anin-video information provider may be providing the Info functioninformation to the first user's second screen, and the Locations andMusic functions to the second user's second screen. In this instance thefirst user causes the transmittal of a request to the services providerto provide the Info write-up to the second screen of the second user.

FIG. 11C is an illustration of a use of a touch screen to definespecific casting/mirroring of second screen content. The previouslyincorporated U.S. patent publication 20130007793 discloses: “A methodand system for generating a dynamic user interface on a second screencontrol device for controlling the content being displayed on primaryviewing screen.” The disclosure is one example of underlying touchscreen technologies that are incorporated herein into a multi-usermulti-second screens embodiments.

FIG. 11C depicts an instance in a second screen 1151 where a first userhas enabled the Info second screen function 1161. In this example,during a playing of the motion picture Casino Royal, an Info functioninformation for the Vesper Martini depiction provides a title 1162, arecipe write-up 1163, and an external link 1164 to additionalinformation. A review later function control 1165 enables the user tosave the item for later review and/or the previously disclosed Shoppingfunction.

In the example of FIG. 11C, two complementary and/or alternative“casting/mirroring” methodologies are implemented. First, the interfaceprovides a set of arrows each associated with a specific transmissionfunction. The up arrow 1171 when touched causes the Info function title1162 and write-up 1163 to be casted/mirrored to the primary screen. Thedown arrow 1174 when touched causes the Info function title 1162,write-up 1163, and link 1164 to be saved to a user specified file,memory, device, and/or location. In this instance, two right arrowsenables sending the Info function title 1162, write-up 1163, and link1164 to a second screen of either one of both of two users, eachassociated with an image/icon and name/handle/avatar. User “Billy” beingin the same room as the user, and user “Jenn” being remotely locatedfrom the user.

The second methodology, directionally similar to the first, enables theuser to cause the activation of the individual transmission functions byswiping 1181 the desired specific content, (e.g., the write-up 1163) or,by for example, swiping an all function control 1191 (e.g., gift boxicon) which would transmit the Info function title 1162, write-up 1163,and link 1164. As in the arrow implementation, a directional swiping(e.g., up, down, left, right, and angled) directs the information to aspecific destination. Touching a messaging function control (e.g., emailfunction icon 1192) would enable the transmission of the information bya particular preselected function. Similarly, icons for the varioussocial media platforms (e.g., Facebook Like icon 1193) may beimplemented to enable the user to automatically post content. Twofingers swiping towards the right may be associated with either sendingall the information or sending the particular information to all otherusers. As everywhere else herein, user preestablished preferences maydefine what specific content may be transmitted to which users. Further,the entire second screen experience of a user may be mirrored on theprimary or second screen of another user.

In those embodiments relying in an acoustic fingerprint synchronizationmethodology, it may be advantageous to provide a Sync function control1199 that when activated by the user would activate the appropriateacoustic fingerprint synchronization routines.

Second Screen Embodiments

Second screen embodiments may be broadly categorized as comprisingeither: (i) item notification displayed together with video playback onthe primary screen that is software synchronized with informationdisplayed on a second screen; (ii) video playback on a primary screenthat is software synchronized with information displayed on a secondscreen; (iii) video playback on a primary screen that is usersynchronized with information displayed on a second screen; or (iv)video playback on a primary screen that is not synchronized withinformation displayed on a second screen.

FIG. 12 is a flow chart of a method for displaying second screeninformation synchronized with video and notification display on aprimary screen. This particular embodiment is dependent on a set ofsoftware applications that are capable of superimposing content on thevideo displayed on the primary screen, and capable of communicatingsynchronizing information from the primary screen to a secondary screenin order to synchronize the display, on the second screen, of responsiveinformation and content.

In an exemplary embodiment, the hardware platform comprises a personalcomputer retrieving video from a streaming video service provider (e.g.,Google streamed video) and displaying the video on a primary screen.Software routines in the personal computer obtain synchronizinginformation (e.g., timing information such a current location time code)from the video as the video is played. The software applicationcomprises routines that have access to video playback functions thatprovide the required synchronizing information (e.g., time codefeedback). For example, in a PC embodiment playing a Google streamedvideo, the getCurrentTime function returns the time code of the currentplay position. Similar time code retrieval functionality is availablewith the Microsoft Media Player, and most software media players.

The video map, information and content related to a corresponding video,play routines specific to the video, and control codes for automaticallyconfiguring or controlling the functions of the end-user devices may bedownloaded, processed, and/or queried at a device/primary screen, and/ordownloaded, processed, and/or queried at a second screen device (e.g., atablet). Responsive to the utilization of the video map, the informationand content displayed on the second screen may be retrieved by anotherdevice (e.g., PC) from the video map or an external source (e.g., remoteserver) and provided to the second screen, and/or directly obtained bythe second screen from the video map or an external source. With respectto a playing of a video, the Video Map ¶å0063-0076, Bookmark GeneratingData ¶å0105-0114, Seek/Step Data ¶å0115-0148, Map Synchronization¶å0149-0188, and the Multi-Navigators ¶å0189-0208 disclosures of U.S.patent publication 20140219630 are incorporated herein by reference.

Referring to FIG. 12, typically, a user would have preestablishedcontent preferences 1201 prior to playing a video. Following userselection of a video or a plurality of videos, the video ID(s) areobtained 1202 to determine if a map and other data is available for thevideo(s) 1203. If a map is not locally available 1203, or if updating ofmaps is automatically performed at this point, then the map isdownloaded from a remote source and processed 1204 (e.g., merged withlocal user database and/or updating). If a map is locally available 1203or following download 1204, the map is retrieved 1205 to display theavailable features 1206. The user's presentation preference is obtained1211, and content preferences if not previously obtained are obtained1212. The presentation preference and content preferences are applied tothe video map to generate a video presentation 1214 (e.g., a playlist ofsegment definitions defining video segments from within the video). Ifvideo playback function preferences were preestablished, the user'sfunction preferences are obtained 1213 and the video presentation isplayed 1215. A user's function preferences (e.g., second screen functiondisplays) may be preestablished, established, and/or modified at anytime (e.g., contemporaneously with preestablishing content preferences,prior to playing a video, prior to playing a feature, and during theplaying of a presentation).

If item notification routines have been activated 1221 in notificationmode and/or second screen mode, then the current play location within avideo 1222 is identified. Once a play location is identified, the itemdata is queried (e.g., a plurality of segment definitions are searched)to identify segment definitions that are responsive to the play locationand the particular function(s) activated 1223. If no segment definitionis responsive, then the play location is continued to be monitored 1222.If a segment definition is responsive 1223, an indication that iteminformation is available for an item being depicted is displayed and/orthe second screen displays are activated 1224. If a request is notreceived from a user for item information 1230 in response to thedisplay of an indication that item information is available 1224, then,if the function notification mode is still activated 1221, the playlocation is continued to be monitored 1222.

In an exemplary embodiment, if a user has activated one or more of thein-video functions (e.g., Who, Locations, Plot Info, and Filmmaking) innotification mode, then, during the playing of a video, an indication isdisplayed that item information is available for an item being currentlydepicted with respect to one or more categories of content of interestto the viewer. The indication that item information is available may bedisplayed on the primary screen and/or the second screen as previouslydetailed. Item notification offer a number of advantages including, forexample, a better match between a request for item information for aparticular item being depicted and the availability of correspondingitem information on a second screen. To some users, the itemnotification enhancements disclosed herein offer a better userexperience with obtaining item information for an item being depicted inone or more of a plurality of informative and entertaining playbackfunctions. Nonetheless, some in-video functions do not requirenotification on a primary screen, or a user may not desire the displayof notifications on the primary screen. Thus, advantageously, itemnotification on the second screen provides an unobtrusive methodology.

If a request is received from a user for item information 1230 inresponse to the display of an indication that item information isavailable 1224, then the playing of the video is automatically paused1231 if not already paused. A user may request item information by, forexample, tapping a corresponding function control on the second screen.In those instances where the second screen comprises remote controlfunctionality, double tapping function control on the second screen willalso automatically pause the playing of the video on the primary screen.Alternatively, the pausing of the video 1231 may be optional as a usermay desire that the video continue playing while the item identificationroutines dynamically respond to changes in the current play location. Auser may prefer to obtain contemporaneously updated frame accurate iteminformation as the video continues to be played. In such cases, userpreference would disable the routines from automatically pausing aplaying of the video. In the case of, for example, the Who function,where item notification on the primary screen is unnecessary or notdesired, item information (e.g. performer/character identification) maybe contemporaneously displayed on the second screen as the videocontinues to be played on the primary screen.

In a second screen application, a user is likely to utilize thecapabilities of the second screen to request item information. Forexample, pressing an item information control button/object in thesecond screen user interface would be received, during a playing of avideo, as a request from a user for item information relating to an itemdepicted within the video. For example, as detailed with respect to theLocations function, pressing the Locations button/object causes theactivation of the display of Locations information notificationroutines, and, when available, the display of locale item information.If locale item information is not available for the current playposition, notification may provide a time to the next instance ofresponsive Locations function information.

In the case of, for example, the Who function, where item notificationon the second screen is unnecessary or not desired, item information(e.g. performer/character identification) may be contemporaneouslydisplayed on the second screen as the video continues to be played onthe primary screen. Nonetheless, the Who function information may alsoinclude, for each of the currently depicted characters. a countdowntimer to the next clip in which the character is present or thedepiction of the character is noteworthy.

In a second screen application, a user is likely to utilize thecapabilities of the second screen to request item information. Forexample, pressing an item information play control button/object in thesecond screen user interface would be received, during a playing of avideo, as a request from a user for item information relating to an itemdepicted within the video. For example, as detailed with respect to theLocations function, pressing the Locations button/object causes theactivation of the display of Locations information routines, and thedisplay, when available of locale item information.

Following the request for item information 1230, a request location(e.g., time code, frame identifier, or some other indicator of the placeor location, within the video that is currently being played) isidentified 1232 that is responsive to the request for item information.A location within the video may be identified by a variety of meansincluding, for example, time codes in a seconds and milliseconds formator an HH:MM:SS:FF (Hours, Minutes, Seconds, and Frames) format, frameidentifiers, bit positions, logical positions identifiers, or any formator other indicator that can consistently accurately identify a locationin a video. A location may also be referred to as a position or place.In a multi-video embodiment, the identification of the video beingcurrently played is particularly material in those situations where thepresentation comprises the playing of segments retrieved from within aplurality of videos. The video may be identified by any of a number ofmethodologies being currently practiced or known in the art.

Dependent on the particular embodiment, to allow for delays by the userto request item information, the identification of the request location1232 may be responsive to a preestablished delay adjustment preference,such as, a user's specific previously defined amount of time, e.g. 10seconds. Herein, whether a user's preference or a system's preferencepreestablished means that the amount is not dependent on a user's actionat the time of the request. Preestablished does not necessarily meanthat the preference is not responsive to the particular segment beingplayed. In those instances that the playing of the video is voicecontrolled, separate default values may be established to accommodatethe fact that user may be more likely to say “where” or “why” fasterthan to be able to press a button on a remote control specially whileviewing a full length motion picture. To facilitate voice response audiocommands may be adopted such as “cpwho”, “cpwhat”, “cpplotinfo”, and“cpinfo”. Alternatively, a user may teach the player a user preferredset of voice commands. In one embodiment, rather than a user configuringthe system at the time of installation or a first playing, autoactivesoftware routines would learn the user's preferences from the user'sactions or commands, and adjust the various settings accordingly.

Once a request location 1232 is identified, the item data is queried(e.g., a plurality of segment definitions are searched) to identifysegment definitions that are responsive to the request location and theplayback functions that are activated 1233. A segment definition isresponsive when it includes the request location, as may be adjusted,within the beginning and ending of the defined segment and is associatedwith information that is responsive to an activated function. A requestlocation, as may be adjusted, matching either the beginning point, theending point, or any point in between is deemed to be within thebeginning and ending. In one embodiment, identifying a segmentdefinition that is responsive comprises, for example, comparing the timecode of the request location and finding a segment definition having abeginning time code that is the same or earlier than the time code ofthe request location, and having an ending time code that is the same orlater than the time code of the request location.

If a responsive segment definition is found 1233, then item informationassociated with a responsive segment definition is provided 1235 (e.g.,displayed, played, and/or retrieved). To accommodate those situations,in which at the time that the user has requested item information, asegment definition is not responsive, the request location may beadjusted to include the request location and a predetermined play periodprior to the request location, for example, 30 seconds. In this fashion,a segment definition may be responsive to the information request or apredetermined play period prior to the request location. In determiningif a segment is responsive to the request location or a predeterminedplay period prior to the request location, the segment definition maybeadjusted and/or the request location/period may be adjusted.Advantageously, this method enables the retrieval and display of iteminformation associated with a number of recently played depictions ofitems having item information available. The item identifyinginformation, such as an item image and descriptors, that are retrievedand displayed enable identification by the user.

Item information may have been already retrieved and loaded intoprocessing memory, and, thus, retrieval is from the memory.Alternatively, the item information is retrieved from any of a varietyof storage memories, local devices, or remote locations as needed.Herein, the term “retrieve” should be understood in the broadest senseand comprises obtain, acquire, procure, download, transfer, extract, andto come into possession by any means or methodology from a local and/orremote source.

If, for example, a user requests item information for an item that wasno longer being depicted, a segment definition may not be found that isresponsive to the request location 1233. In such situations, a propermessage is displayed (e.g., item information not available for thislocation). Alternatively, a number of methodologies may be implementedto provide item information 1235 for a most recently depicted item forwhich item information is available 1234. For example, segmentdefinitions are searched to identify a segment definition that has anending point prior to and closest to, within certain parameters, therequest location.

If a user requests additional item information 1240 relating todisplayed item information, then the display of additional iteminformation is enabled 1241 in response to the request for additionalitem information. It is understood that the retrieval, providing, anddisplaying of additional item information may be subject to the controlof a third party provider or a source not within the control of theuser's apparatus. The term herein “enabling a display” does notguarantee or require that in fact additional item information isretrieved, provided and/or displayed. For example, item information maybe linked to a remote source that customarily provides additional iteminformation but the source is not accessible at the particular time ofthe user request for additional item information. Such situations stillsatisfy the “enabling a display”. Additional item information comprises,for example, information specific to the purchasing of the item, atransfer to a website's one click page, content previously downloaded,and content retrieved from within the video or from within a pluralityof videos. For example, additional item information may providedepiction of the item extracted from a full-length movie franchise.

A user may request additional item information using a variety oftechnologies and methodologies. For example, clicking or pressing anitem information play control button/object in an user interface on thesecond screen would be received, during the display of item information,as a request from a user for additional item information relating to asingle item for which item information is displayed. Where iteminformation is displayed for multiple items, selection of a desired oneof the displayed item information would be required in connection with,for example, the clicking or pressing of an item information playcontrol button/object on the second screen. For example, the displayeditem information previously detailed each constitute a button/objectassociated with a “clickable” event that when activated or selected bythe user would constitute an additional item information request.

The display of multiple item information may be limited to a system oruser preestablished number of the last depicted items prior to therequest for item information. In an exemplary embodiment up to five iteminformation are displayed for items that were depicted within 30 secondsof the item information request, with the most recently depicted itembeing displayed to the right of the others.

Termination of the display of additional item information 1242 returnsto the screen/menu providing multiple item information 1235 rather thanautomatically resume video play 1251. This would be particularlyadvantageous where multiple item information is displayed.Alternatively, termination of the display of additional item information1251 automatically resumes video play if previously paused. Atermination of the display of additional item information may take manyforms, such as exiting the additional information function (e.g. exitinga website, and exit or escape request). Any terminating event thatindicates that the user desires to resume video play may be considered avideo play request.

If a user does not request additional item information relating todisplayed item information 1240, then a resume video play request ismonitored and if received 1251, display of the item information isterminated and play of the video is resumed responsive to the iteminformation request location. At this point, unless terminated byanother process, the item identification routines, subject to thefunction notification mode 1221, restart at the appropriate one of themonitoring of a play location 1222 or monitoring of a request for iteminformation 1230. If the resume video playback is terminated 1251, theprocess may end, or restart at any suitable point (e.g., display feature1206, or obtain video ID 1202) responsive to the terminating action(e.g., a new video is detected).

When the user elects to exit the additional information environment,e.g., a page in a website, the playing of the video may be automaticallyresumed at a location that is responsive to the request location.Resuming a playing of the video responsive to the request locationfollowing a termination of a displaying of additional item informationdoes not require resuming a playing of the video at the exact locationat which the request for item information was received. The resumptionlocation may be additionally responsive to other considerationsincluding, for example, technical limitations. In an exemplaryembodiment, rather than resuming a playing of the video at a locationthat is responsive to the request location, a playing of the video isresumed at a location that is also responsive to one or moredeterminants, such as, the amount of time that the playing of the videohas been paused, the beginning or ending of the segment depicting theitem, and/or a beginning of a clip or scene. The suitable location forresuming the playing of the video may be directly identified by asegment definition that is responsive to the request location. Further,the automatic resuming of the playing of the video may itself beresponsive to the number of item for which item information isdisplayed. For example, if item information is displayed for multipleitems, then the playing of the video would not be resumed following atermination of a displaying of additional item information. Thus,resuming a playing of the video at a location that is responsive to therequest location is not limited to resuming a playing of the video atthe request location.

Advantageously, an information mode may be selected by the user thatautomatically pauses the playing of the video and provides responsiveinformation consistent with the enabled playback function as thepresentation is played. To accommodate such a mode the map may furtherassociate with the segment definition a time deemed adequate for theplayback to be paused and automatically resumed. Alternatively, theresumption of the playback would be responsive to a user playbackresumption request. Still further, since many of the presentations inthe Presentation feature fall within certain standardized categories, auser may preestablish combinations of categories of presentation (e.g.,Intellectual) with one or more playback functions (e.g., Filmmaking andPlot Info).

The flow chart of FIG. 12 has been intentionally drafted to disclose anumber of possible embodiments. A number of permutations, combinations,substitutions, and/or additions in the specific steps, and recursivefunctions may be initiated at different points in the process than thoseillustrated. In an exemplary second screen embodiment, the itemindication 1224, item information 1235, and/or additional iteminformation 1241, elements of in-video functions are displayed on thesecond screen. Further, the playing of the video may or may not bepaused responsive to the particular primary and second screen hardwareplatform, the particular in-video functions that are active, and userfunction-specific preferences.

Further, the detailed disclosure herein offer additional elements thatmay be synergistically, additionally or alternatively, implemented inone of many possible second screen embodiments. The inventions are notlimited to any particular embodiment. Many combinations of the variouselements are possible to fit each of a multitude of situations. In anexemplary embodiment a video map comprises, for example, the informationto support a user's version preferences, presentation preferences (e.g.,Presentations, Compilations, Subjects, Dilemmas, Best Of, Performers,Shopping, Music, Search, and Preview), content preferences, in-videopreferences (e.g., Locations function), and playback functions (e.g.,What function). Accordingly, a second screen embodiment that is softwareintegrated and synchronized with a primary screen display performs thesteps consistent with such a full featured video map.

Clearly, a most preferred embodiment is not limited to the simplifiedembodiment illustrated above. FIG. 12 details some of the extensivefunctionality that is provided by the multiple playback functionspreviously detailed. Further, while the content preference andpresentation preference steps precede the playback functions in FIG. 12as would be the case in some situations, it is not always the case. Forexample, as previously indicated, an adjustment of content preferencesmay take place at any time during video playback.

A combination of a presentation preference and a function preference isparticularly advantageous in providing a highly customized and enhancedmovie experience for, specially, a narrowly targeted audience. Forexample, the combination of an “Intellectual” presentation that focuseson the plot points of a movie (e.g., Cloud Atlas) together with the PlotInfo function enabled on a second screen offers material advantages tothe user that desires to focus on the intellectual aspects of the movierather than on, for example, the action clips. The enablement of thePlot Info function provides the additional intellectual insight thatthat user would prefer.

An example of a highly tailored second screen embodiment is the “X-MenMovies Cerebro App” which claims to permit a user to: “Experience theX-Men: Days Of Future Past on Second Screen”. The application syncs theplaying, on a Blu-ray player, of the Blu-ray release of the motionpicture X-Men: Days Of Future Past to a user's tablet or smartphone. Theapplication also enables a user to “Flick Movie Extras” from the user'sdevice to the TV screen. To set-up the application, the user is requiredto: Download the X-Men Movies Cerebro App to the second screen device;Start the movie on the Blu-ray player; and Sync the app to the movie viaWi-Fi or manual connection. To access the additional extras the user isinstructed to “tap the blue dot in the center 3 times on the main menuof the X-Men Movies Cerebro App.” While the Cerebro App is limited interms of the second screen functions it provides, the application'sfoundational routines, methodologies, and software technologies,including automated and manual synchronization of the second screen withvideo playback on the primary screen, are incorporated herein byreference and may be implemented in various embodiments disclosedherein.

FIG. 13 is a flow chart of a method for displaying second screeninformation synchronized with video display on a primary screen. Thisparticular exemplary embodiment is dependent on a software applicationthat is capable of synchronizing the display of information to a videoplaying on a primary screen. This exemplary embodiment is not limited toany particular device playing the video, the source of the video, or anapplication directly or indirectly accessing the video. Rather thisembodiment relies on a software application residing in the secondscreen that is capable of synchronizing the display of information onthe second screen to the playing of a video on a primary screen.

Referring to FIG. 13, typically, a following selection of a video, auser identifies the video being played to the software applicationrunning on the second screen. Alternatively, the video being played isidentified by a methodology similar to how the Shazam mobile applicationidentifies songs. The Shazam website explains that when a user desiresto identify an audible musical item, the user starts the Shazamapplication and taps the Shazam button. The Shazam identificationroutines relies on creating a digital fingerprint of the audio beingplayed and comparing it against a database of digital fingerprintsassociated with information about musical items. Upon a match, the useris provided an identification of the musical item and other relatedinformation including purchase information. When suitable lyrics areavailable, the Shazam application is capable of synchronizing the lyricsto the musical track, providing the user the potential for a Karaokelike experience.

In an exemplary embodiment, as the video is played, the second screenapplication obtains sufficient instances of a representation (e.g.,acoustic fingerprint, digital fingerprint, or audio signature) of theaudio to be able to identify, within a certain time, the video beingplayed 1301. With respect to identifying a particular release of a videoin order to retrieve and apply the appropriate video map, and modifying,responsive to the particular release of a video, a video map's segmentdefinitions, seek/step data, and/or bookmark generating data, the VideoMaps Responsive to Video Releases ¶å0269-0284 disclosures of U.S. patentpublication 20140219630 are additionally incorporated herein byreference. If the video is not identified, an appropriate messageseeking user input may be provided.

In those embodiments, where the video map is processed by the secondscreen 1302, once the video is identified 1301, a search of theavailable video maps is performed 1303 to determine if a video map forthe identified video is currently stored in the second screen. If a mapis not locally available 1303, or if updating of maps is automaticallyperformed at this point, then the map is downloaded from a remote source1304 and processed 1305 (e.g., merged with local user database and/orupdating). If a map is locally available 1303 or following download1304, the map is locally processed 1305 to display the available secondscreen functions 1311. If a map is not locally processed 1302, then themap is processed by another device or remotely processed 1306 in orderto communicate to, and display the available functions on, the secondscreen 1311.

A user's second screen preferences may be preestablished, established,and/or modified at any time (e.g., prior to or during a playing of thevideo). The function preferences are obtained 1312 to determine if anysecond screen notifications are to be activated 1321.

If second screen notification routines have been activated 1321 innotification mode and/or second screen mode, then the current playlocation within a video 1322 is identified. Once a play location isidentified, the item data is queried (e.g., a plurality of segmentdefinitions are searched) to identify segment definitions that areresponsive to the play location and the particular second screennotification(s) are activated 1323. If no segment definition isresponsive, then the play location is continued to be monitored 1322. Ifa segment definition is responsive 1323, an indication that iteminformation is available is displayed and/or the second screeninformation displays are activated 1324. If a request is not receivedfrom a user for item information 1330 in response to the display of anindication that item information is available 1324, then, if thefunction notification mode is still activated 1321, the current playposition is continued to be monitored 1322.

In an embodiment that does not provide direct current play positionidentification, a number of methodologies may be implemented tootherwise synchronize second screen functionality with a playing of avideo on a primary screen. Generally, with respect to second screensystems and methods, and specifically, with respect to the apparatus,systems, architectures, methods, and functionalities for synchronizingthe display of information on a second screen to a playing of a video ona primary screen, the following disclosures are incorporated herein byreference: U.S. Pat. No. 8,150,165 titled “System And Method For VisualRecognition”; U.S. Pat. No. 8,374,387 titled “Video Entity RecognitionIn Compressed Digital Video Streams”; U.S. Pat. No. 8,516,528 titled“Synchronization Of 2nd Screen Applications”; U.S. Pat. No. 8,516,533titled “Second Screen Methods And Arrangements”; U.S. patent application20110289532 titled “System And Method For Interactive Second Screen”;U.S. patent application 20090327894 titled “Systems And Methods ForRemote Control Of Interactive Video”; U.S. patent application20130111514 titled “Second Screen Interactive Platform”; and U.S. patentapplication 20140071342 titled “Second Screen Content”. Further,currently available open source applications, such as Echonest, providesource code for the implementation of acoustic fingerprintsynchronization.

In a methodology similar to a video identification 1301, a current playposition may be identified 1322 by continuously and/or intermittinglymatching audio samples (e.g., an acoustic signature) to a database ofacoustic data to retain video map synchronization with the playing ofthe video on the primary screen. “An acoustic fingerprint is a condenseddigital summary, deterministically generated from an audio signal, thatcan be used to identify an audio sample or quickly locate similar itemsin an audio database.” (Source: Wikipedia acoustic fingerprint)Intermittent samples of audio (e.g., once every minute or approximatelya total of 100 samples in an average motion picture) may be supplementedby maintaining an expected play clock during the sampling interval.Advantageously, the methodology may reduce the amount of acousticsignature data in the video map while still providing an effectiveidentification of a current play position within the necessary precisiontolerances. Further, the methodology enables materially reducing thefrequency of audio sampling responsive to the second screen functionsthat are activated and user actions. For example, if a user has onlyenabled the Plot Info second screen function, which averages 15instances, maintaining video map and video playback synchronization mayonly require matching 20-25 audio samples.

In an acoustic fingerprint embodiment, a second screen device receivingsynchronizing information that is responsive to a video playing on aprimary screen device may comprise, for example, the second screendevice's microphone capabilities receiving (e.g., listening to) an audiothat is provided by a playing of the video, and any ambient, background,atmospheric audio, sounds, and noises that are present. Thus,synchronizing information that is responsive to a video playing on aprimary screen device may also be responsive to other audio, sounds, andnoises that are present. Within the confines of a household, thegeneration of an acoustic fingerprint and the matching to a database todetermine a current play position readily compensates for extraneoussounds and noises. As a backup, a second screen device receivingsynchronizing information comprises a user manually inputtingsynchronizing information into the second screen device.

In an exemplary embodiment, if a user has activated one or more of thein-video functions (e.g., Who, Locations, Plot Info, and Filmmaking) innotification mode, then, during the playing of a video, an indication isdisplayed 1324 that item information is available for an item beingcurrently depicted with respect to one or more categories of content ofinterest to the viewer. Item notification offer a number of advantagesincluding, for example, a better match between a request for iteminformation for a particular item being depicted and the availability ofcorresponding item information on a second screen. To some users, theitem notification enhancements disclosed herein offer a better userexperience with obtaining item information for an item being depicted.Nonetheless, some in-video functions do not require notification on asecond screen, or a user may not desire the display of notifications onthe second screen. In this particular second screen embodiment, itemindication 1324 may additionally or alternatively comprise and audionotification, vibration, and/or increase brightness of the second screenor portion of the second screen.

If a request is received from a user for item information 1330 inresponse to the display of an indication that item information isavailable 1324, the request location is identified 1332 either directlyby, for example, sampling the audio and/or by the internalsynchronization clock. Once a request location 1332 is identified, theitem data is queried (e.g., a plurality of segment definitions aresearched) to identify segment definitions that are responsive to therequest location and the second screen functions that are activated1333. If a responsive segment definition is found 1333, then iteminformation associated with a responsive segment definition is retrievedand displayed 1335. Item information may have been already retrieved andloaded into processing memory, and, thus, retrieval is from the memoryof the second screen. Alternatively, the item information is retrievedfrom any of a variety of storage memories, local devices, or remotelocations as needed. Herein, the term “retrieve” should be understood inthe broadest sense and comprises obtain, acquire, procure, download,transfer, extract, and to come into possession by any means ormethodology from a local and/or remote source.

When a user requests item information for an item that was no longerbeing depicted, a segment definition may not be found that is responsiveto the request location 1332. In such situations, a proper message isdisplayed (e.g., item information not available for this location).Alternatively, a number of methodologies may be implemented to provideitem information 1335 for a most recently depicted item for which iteminformation is available 1334. For example, segment definitions aresearched to identify a segment definition that has an ending point priorto and closest to, within certain parameters, the request location.

If a user requests additional item information 1340 relating todisplayed item information, then the display of additional iteminformation is enabled 1341 in response to the request for additionalitem information. Termination of the display of additional iteminformation 1342 returns to the screen/menu providing multiple iteminformation 1335. This would be particularly advantageous where multipleitem information is displayed. Alternatively, termination of the displayof additional item information 1341 restores the function notificationactivation routines 1321. A termination of the display of additionalitem information may take many forms, such as exiting the additionalinformation function (e.g. exiting a website, and exit or escaperequest). If a user does not request additional item informationrelating to displayed item information 1340, then the functionnotification activation routines 1321 are restored, unless the userelects to exit the application 1351.

The flow chart of FIG. 13 has been intentionally drafted to disclose anumber of possible embodiments. A number of permutations, combinations,substitutions, and/or additions in the specific steps, and recursivefunctions may be initiated at different points in the process than thoseillustrated. In an exemplary second screen embodiment, the itemindication 1324, item information 1335, and/or additional iteminformation 1341, elements of in-video functions are displayed on thesecond screen.

It is noted that while certain exemplary embodiments are detailed,directly or by incorporation, for certain second screen functions, theinventions are not limited to the exemplary data elements utilized inany function or specific embodiment. An advantage of a particular secondscreen embodiment, feature, or element is not necessarily limited tothat particular second screen embodiment, feature, or element. As theexemplary embodiments demonstrate, a method or step that is disclosedwith respect to a certain second screen embodiment may be similarlyimplemented in other second screen embodiments. Many variations of theelements included in a particular second screen embodiment are possibleby way of permutation, combination, substitution, deletion, and/oradditions. A feature or in-video function may be enhanced by theaddition of data elements (e.g., segment definitions, frameidentification, play position identification, image and thumbnailidentifiers, write-ups, and linkages) detailed with respect to otherfeatures or in-video functions. A video map addressing the variousfeatures and functions detailed herein provides a great database ofadditional information that may be mined to support the features,in-video functions, second screen functions, and playback functions. Theapplicability of the “and/or” concept is not limited to the explicitinstances where the terms “and/or” are used. The explicit use of theterms “and/or” serves to underscore the available permutations.

The teachings disclosed herein, directly and indirectly by, for example,incorporation, are intended to show a variety of architectures,services, capabilities, systems, methods, and inventive elements whichare combined and may be combined to suit particular embodiments. While afunction of incorporation is to provide additional detail explanation,the synergies among and between the various inventive elements is asignificant feature of and object of incorporation. The incorporation byreference at a specific place within the specification is not intendedto limit the extent to which the reference is incorporated, or themanner in which it may be integrated. Where a teaching may be deemed tobe at cross purposes, or otherwise incompatible, with some otherteaching, it ought to be understood as a possible alternative to beutilized as a particular preferred embodiment may require.

While elements of the inventions have been detailed in conjunction withspecific embodiments thereof, it is evident that many alternatives,modifications and variations are possible and will be apparent to thoseskilled in the art in light of the foregoing descriptions. Accordingly,it is intended to embrace all such alternatives, modifications,variations, and combinations as fall within the spirit and broad scopeof the specification. The teachings that have been cited andincorporated herein are offered by way of example, and not limitation,of the underlying foundation of knowledge and skill that is available toa person of ordinary skill in the art. Many of the features, components,and methods found in the art may be incorporated, as suggested herein,in a preferred embodiment; and since other modifications and changesvaried to fit particular requirements and environments will be apparentto those skilled in the art, the inventions are not limited to theembodiments set forth or suggested herein. It is to be understood thatthe inventions are not limited thereby. It is also to be understood thatthe specific details shown are merely illustrative, and that theinventions may be carried out in other ways without departing from thebroad spirit and scope of the specification.

What is claimed is:
 1. A video information display method comprising:downloading, over a communications network and to a memory of a secondscreen device, a subset of subtitles, the subset of subtitlescorresponding to preselected portions of a dialogue of a video;receiving audio information principally from a playing of the video on aprimary screen device; generating an acoustic fingerprint responsive tothe received audio information; establishing, responsive to the acousticfingerprint, a synchronization to the playing of the video on theprimary screen device; and displaying the subset of subtitles on thesecond screen device responsive to a time period offset synchronizationto the video playing on the primary screen device, the time periodoffset synchronization being responsive to a user's time period offsetpreference, whereby the display of the subset of subtitles on the secondscreen device is intermittent and lags a contemporaneous playing of thevideo on the primary screen device.
 2. A video information displaymethod comprising: downloading, over a communications network and to amemory of a second screen device, supplementary information; receivingsynchronizing information that is responsive to a playing of a video ona primary screen device; establishing, responsive to the synchronizinginformation, a synchronization to the playing of the video on theprimary screen device; and displaying the supplementary information onthe second screen device responsive to an offset synchronization to thevideo playing on the primary screen device, whereby the display ofsupplementary information on the second screen device lags acontemporaneous playing of the video on the primary screen device. 3.The method of claim 2, wherein the supplementary information comprisestextual information representative of dialogue of the video.
 4. Themethod of claim 2, wherein the supplementary information comprises asubset of subtitles corresponding to preselected portions of a dialogueof the video, and wherein the display of supplementary information isintermittent and lags a contemporaneous playing of the video.
 5. Themethod of claim 2, wherein the supplementary information comprisesclosed captioning.
 6. The method of claim 2, wherein the supplementaryinformation comprises textual information representative of preselectedportions of a dialogue of the video, and whereby the display of textualinformation is intermittent.
 7. The method of claim 2, wherein thesupplementary information comprises subtitles.
 8. The method of claim 2,wherein the offset synchronization is responsive to a default timeperiod delay.
 9. A second screen device comprising: a communicationinterface for downloading, over a communications network and to a memoryof the second screen device, supplementary information, and forreceiving, from a primary screen device, an identification of a playposition in a video playing on the primary screen device; a processorfor establishing, responsive to the identification of the play position,a synchronization to the playing of the video on the primary screendevice; and a display screen for displaying the supplementaryinformation responsive to an offset synchronization to the video playingon the primary screen device, whereby the display of supplementaryinformation on the second screen device lags a contemporaneous playingof the video on the primary screen device.
 10. The device of claim 9,wherein the supplementary information comprises textual informationrepresentative of dialogue of the video.
 11. The device of claim 9,wherein the supplementary information comprises a subset of subtitlescorresponding to preselected portions of a dialogue of the video, andwherein the display of supplementary information is intermittent andlags a contemporaneous playing of the video.
 12. The device of claim 9,wherein the offset synchronization is responsive to a default timeperiod delay.
 13. The device of claim 9, wherein the supplementaryinformation comprises closed captioning.
 14. A second screen devicecomprising: a communications interface for downloading, over acommunications network and to a memory of the second screen device,supplementary information; an integrated microphone for receiving audioinformation principally from a playing of a video on a primary screendevice; a processor for generating an acoustic fingerprint responsive tothe received audio information, and for establishing, responsive to theacoustic fingerprint, a synchronization to the playing of the video onthe primary screen device; and a display screen for displaying thesupplementary information responsive to an offset synchronization to thevideo playing on the primary screen device, whereby the display ofsupplementary information on the second screen device lags acontemporaneous playing of the video on the primary screen device. 15.The device of claim 14, wherein the supplementary information comprisestextual information representative of dialogue of the video.
 16. Thedevice of claim 14, wherein the supplementary information comprises asubset of subtitles corresponding to preselected portions of a dialogueof the video, and wherein the display of supplementary information isintermittent and lags a contemporaneous playing of the video.
 17. Thedevice of claim 14, wherein the offset synchronization is responsive toa user's time period delay preference.
 18. The device of claim 14,wherein the supplementary information comprises closed captioning. 19.An information provider system comprising: a memory storingsupplementary information and offset synchronization information; acommunications interface for downloading, over a communications networkand to a second screen device, supplementary information and offsetsynchronization information; a processor for processing a request, froma second screen device, for supplementary information, and for executinga downloading, to the second screen device, of the supplementaryinformation and the offset synchronization information to enable thesecond screen device to display the supplementary information responsiveto an offset synchronization to a video playing on a primary screendevice, whereby the display of supplementary information on the secondscreen device lags a contemporaneous playing of the video on the primaryscreen device.
 20. The system of claim 19, wherein the supplementaryinformation comprises subtitles, and wherein the offset synchronizationis further responsive to a user's time period offset preference.